Andy Glew's comp-arch.net wiki, http://semipublic.comp-arch.net

If you are reading this elsewhere, e.g. at site waboba.info, it is an unauthorized copy, and probably a malware site.
comp-arch.net wiki on hold from October 17, 2011

From CompArch

A processor that is specialized to performing the calculations of 2D and 3D graphics.

Originally very much fixed function with very specialized hardware that could not be used for much other than graphics, circa 2004-2009 GPUs with the advent of programmable shaders became much more versatile. Certainly usable for many HPC or computation intensive workloads. This led to the GPGPU phenomenon.

GLEW OPINION: The GPGPU phenomenon happened mainly because general purpose microprocessor companies dropped the ball. They knew what they needed to do to compete with GPUs, but did not want to spend silicon. I remember one discussion where somebody said "in order to provide the same bandwidth we would need to build 64 FP multiplers. We can't afford that." I.e. microproceessor vendors did not realize that they were in the business of selling compute cycles. Conversely, the GPGPU phenomenon arose as a threat to CPUs because (1) graphics had an almost insatiable demand for compute cycles (2) but in the early days, graphics needed only simple microarchitectures, so brute force sufficed but, however (3) over time graphics wanted more and more flexibility in programmable shaders, etc. - until eventually GPUs became very flexible compute engines. I.e. they evolved from brute force, simplified, stripped down, fixed function devices to flexibly programmable devices. From integer and fixed point only to floating point. From irregular memory hierarchies with no cache, to cache hierarchies. It remains to be seen whether GPUs will evolve to support all of the baggage of CPUs.

Apart from fixed function graphics hardware, the most interesting aspects of GPUs from the point of view of general purpose computation are

• the array of programmable shader compute engines: many, highly parallel, highly multithreaded, CPU-style SIMD or SIMT coherent threading, with thread scheduling ranging from simple to complicated dynamic
• texture sampling units