The RFC2616 (HTTP specification), § 1.3, Terminology defines:

A program that establishes connections for the purpose of sending requests.
user agent
The client which initiates a request. These are often browsers, editors, spiders (web-traversing robots), or other end user tools.

Thus user agent is the name given to all programs that may be used to access websites, e.g. Web browsers.

That same specification defines (§ 14.43) the User-Agent header as that which "contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations", and states that it should be sent. Most modern web browsers include this field along with their requests.

User-agent string is the terminology usually used for the value sent by user agents to identify themselves.

An example user-agent string is the following:

Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/2009042813 Gentoo Iceweasel/3.0.10

Which may be divided across serveral parts:

the codename of the application. Most user-agent strings begin that way, mainly for historical reasons, when some website developers banned user agents other than Netscape Navigator.
the application version. This first part can be read as "Netscape Navigator 5.0", that is, those versions of Netscape that use the Gecko rendering engine.
the graphical system used for drawing the browser interface.
the browser supports 128-bits or higher.
Linux x86_64
the operating system in which the user agent is running.
the language of the user agent interface.
CVS Branch tag, rendering engine version.
the rendering engine used and its build date.
the user agent vendor.
the user agent name and version number.

Another example may be:

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0)

Mozilla/4.0 compatible
is supposed to mean "compatible with Netscape Navigator 4.0."
MSIE 8.0
the name of the user agent. Stands for Microsoft® Internet Explorer® 8.0.
Windows NT 6.1
operating system. It is the codename for Microsoft® Windows® 7.
stands for "Windows on Windows 64-bit". Means that this is the 32-bit version running on a 64-bit OS.
the rendering engine used and its version.

Note: Windows® operating system, Internet Explorer® internet browser and Microsoft® are registered trademarks of Microsoft Corporation in the United States and/or other countries. Gentoo® is a trademark of Gentoo Foundation, Inc.. Linux® is a trademark of Linus Torvalds in the United States and/or other countries.

For further information, please read the Wikipedia article on user agent.