performance wall of the parallelized sequential computing - Janos Vegh - Bog

The development of computing performance faced different limitations. The "memory wall" limited the performance from the beginning. For today the single processor performance has hit the "complexity wall" and "thermal wall". The computing approached and reached the final limits the laws of the nature enable. The practical need for increasing the computing performance (large amount of computing operations and data, the growing interest in deploying artificial intelligence; all these in real-time) gave more focus to parallelization. The discussion in this booklet scrutinizes whether the present efforts to produce high-performance systems from a very large number of parallelized sequential processors can be successful. A new limitation: the "performance wall" is faced, which originates from the very basics of the computing: the computing paradigm and its clock-driven electronic implementation. This finding has far-reaching consequences. The book explains the sometimes shocking unexpected saturation limitations and failures in fields like building exa-scale supercomputers, running exa-scale programs, or simulating the operation of the human brain.