multithreading computer architecture this article describes hardware supports for multithreads for thread in software see thread computer science a process with two threads of execution running on a single processor in computer architecture multithreading is the ability of a central processing unit cpu or a single core in a multicore processor to execute multiple processes or threads concurrently supported by the operating system this approach differs from multiprocessing in a multithreaded application the processes and threads share the resources of a single or multiple cores which include the computing units the cpu caches and the translation lookaside buffer tlb where multiprocessing systems include multiple complete processing units in one or more cores multithreading aims to increase utilization of a single core by using threadlevel parallelism as well as instructionlevel parallelism as the two techniques are complementary they are sometimes combined in systems with multiple multithreading cpus and with cpus with multiple multithreading cores overview the multithreading paradigm has become more popular as efforts to further exploit instructionlevel parallelism have stalled since the late 1990s this allowed the concept of throughput computing to reemerge from the more specialized field of transaction processing even though it is very difficult to further speed up a single thread or single program most computer systems are actually multitasking among multiple threads or programs thus techniques that improve the throughput of all tasks result in overall performance gains two major techniques for throughput computing are multithreading and multiprocessing advantages if a thread gets a lot of cache misses the other threads can continue taking advantage of the unused computing resources which may lead to faster overall execution as these resources would have been idle if only a single thread were executed also if a thread cannot use all the computing resources of the cpu because instructions depend on each others result running another thread may prevent those resources from becoming idle disadvantages multiple threads can interfere with each other when sharing hardware resources such as caches or translation lookaside buffers tlbs as a result execution times of a single thread are not improved and can be degraded even when only one thread is executing due to lower frequencies or additional pipeline stages that are necessary to accommodate threadswitching hardware overall efficiency varies intel claims up to 30 improvement with its hyperthreading technology while a synthetic program just performing a loop of nonoptimized dependent floatingpoint operations actually gains a 100 speed improvement when run in parallel on the other hand handtuned assembly language programs using mmx or altivec extensions and performing data prefetches as a good video encoder might do not suffer from cache misses or idle computing resources such programs therefore do not benefit from hardware multithreading and can indeed see degraded performance due to contention for shared resources from the software standpoint hardware support for multithreading is more visible to software requiring more changes to both application programs and operating systems than multiprocessing hardware techniques used to support multithreading often parallel the software techniques used for computer multitasking thread scheduling is also a major problem in multithreading types of multithreading interleavedtemporal multithreading coarsegrained multithreading the simplest type of multithreading occurs when one thread runs until it is blocked by an event that normally would create a longlatency stall such a stall might be a cache miss that has to access offchip memory which might take hundreds of cpu cycles for the data to return instead of waiting for the stall to resolve a threaded processor would switch execution to another thread that was ready to run only when the data for the previous thread had arrived would the previous thread be placed back on the list of readytorun threads for example cycle i instruction j from thread a is issued cycle i 1 instruction j 1 from thread a is issued cycle i 2 instruction j 2 from thread a is issued which is a load instruction that misses in all caches cycle i 3 thread scheduler invoked switches to thread b cycle i 4 instruction k from thread b is issued cycle i 5 instruction k 1 from thread b is issued conceptually it is similar to cooperative multitasking used in realtime operating systems in which tasks voluntarily give up execution time when they need to wait upon some type of the event this type of multithreading is known as block cooperative or coarsegrained multithreading the goal of multithreading hardware support is to allow quick switching between a blocked thread and another thread ready to run to achieve this goal the hardware cost is to replicate the program visible registers as well as some processor control registers such as the program counter switching from one thread to another thread means the hardware switches from using one register set to another to switch efficiently between active threads each active thread needs to have its own register set for example to quickly switch between two threads the register hardware needs to be instantiated twice additional hardware support for multithreading allows thread switching to be done in one cpu cycle bringing performance improvements also additional hardware allows each thread to behave as if it were executing alone and not sharing any hardware resources with other threads minimizing the amount of software changes needed within the application and the operating system to support multithreading many families of microcontrollers and embedded processors have multiple register banks to allow quick context switching for interrupts such schemes can be considered a type of block multithreading among the user program thread and the interrupt threads interleaved multithreading barrel processor the purpose of interleaved multithreading is to remove all data dependency stalls from the execution pipeline since one thread is relatively independent from other threads there is less chance of one instruction in one pipelining stage needing an output from an older instruction in the pipeline conceptually it is similar to preemptive multitasking used in operating systems an analogy would be that the time slice given to each active thread is one cpu cycle for example cycle i 1 an instruction from thread b is issued cycle i 2 an instruction from thread c is issued this type of multithreading was first called barrel processing in which the staves of a barrel represent the pipeline stages and their executing threads interleaved preemptive finegrained or timesliced multithreading are more modern terminology in addition to the hardware costs discussed in the block type of multithreading interleaved multithreading has an additional cost of each pipeline stage tracking the thread id of the instruction it is processing also since there are more threads being executed concurrently in the pipeline shared resources such as caches and tlbs need to be larger to avoid thrashing between the different threads simultaneous multithreading simultaneous multithreading the most advanced type of multithreading applies to superscalar processors whereas a normal superscalar processor issues multiple instructions from a single thread every cpu cycle in simultaneous multithreading smt a superscalar processor can issue instructions from multiple threads every cpu cycle recognizing that any single thread has a limited amount of instructionlevel parallelism this type of multithreading tries to exploit parallelism available across multiple threads to decrease the waste associated with unused issue slots for example cycle i instructions j and j 1 from thread a and instruction k from thread b are simultaneously issued cycle i 1 instruction j 2 from thread a instruction k 1 from thread b and instruction m from thread c are all simultaneously issued cycle i 2 instruction j 3 from thread a and instructions m 1 and m 2 from thread c are all simultaneously issued to distinguish the other types of multithreading from smt the term temporal multithreading is used to denote when instructions from only one thread can be issued at a time in addition to the hardware costs discussed for interleaved multithreading smt has the additional cost of each pipeline stage tracking the thread id of each instruction being processed again shared resources such as caches and tlbs have to be sized for the large number of active threads being processed implementations include dec later compaq ev8 not completed intel hyperthreading technology ibm power5 sun microsystems ultrasparc t2 cray xmt and amd bulldozer and zen microarchitectures implementation specifics a major area of research is the thread scheduler that must quickly choose from among the list of readytorun threads to execute next as well as maintain the readytorun and stalled thread lists an important subtopic is the different thread priority schemes that can be used by the scheduler the thread scheduler might be implemented totally in software totally in hardware or as a hardwaresoftware combination another area of research is what type of events should cause a thread switch cache misses interthread communication dma completion etc if the multithreading scheme replicates all of the softwarevisible state including privileged control registers and tlbs then it enables virtual machines to be created for each thread this allows each thread to run its own operating system on the same processor on the other hand if only usermode state is saved then less hardware is required which would allow more threads to be active at one time for the same die area or cost computing portal superthreading speculative multithreading a survey of processors with explicit multithreading acm march 2003 by theo ungerer borut robi and jurij silc vtemajor fields of computer sciencenote this template roughly follows the 2012 acm computing classification systemhardware printed circuit board peripheral integrated circuit very large scale integration systems on chip socs energy consumption green computing electronic design automation hardware acceleration computer systemsorganization computer architecture embedded system realtime computing dependability networks network architecture network protocol network components network scheduler network performance evaluation network service software organization interpreter middleware virtual machine operating system software quality software notationsand tools programming paradigm programming language compiler domainspecific language modeling language software framework integrated development environment software configuration management software library software repository software development software development process requirements analysis software design software construction software deployment software maintenance programming team opensource model theory of computation model of computation formal language automata theory computational complexity theory logic semantics algorithms algorithm design analysis of algorithms algorithmic efficiency randomized algorithm computational geometry mathematicsof computing discrete mathematics probability statistics mathematical software information theory mathematical analysis numerical analysis informationsystems database management system information storage systems enterprise information system social information systems geographic information system decision support system process control system multimedia information system data mining digital library computing platform digital marketing world wide web information retrieval security cryptography formal methods security services intrusion detection system hardware security network security information security application security humancomputerinteraction interaction design social computing ubiquitous computing visualization accessibility concurrency concurrent computing parallel computing distributed computing multithreading multiprocessing artificialintelligence natural language processing knowledge representation and reasoning computer vision automated planning and scheduling search methodology control method philosophy of artificial intelligence distributed artificial intelligence machine learning supervised learning unsupervised learning reinforcement learning multitask learning crossvalidation graphics animation rendering image manipulation graphics processing unit mixed reality virtual reality image compression solid modeling appliedcomputing ecommerce enterprise software computational mathematics computational physics computational chemistry computational biology computational social science computational engineering computational healthcare digital art electronic publishing cyberwarfare electronic voting video games word processing operations research educational technology document management book category portal wikiproject commons vteprocessor technologiesmodels turing machine universal postturing quantum belt machine stack machine finitestate machine with datapath hierarchical queue automaton register machines counter pointer randomaccess randomaccess stored program architecture von neumann harvard modified dataflow transporttriggered cellular endianness memory access numa huma loadstore registermemory cache hierarchy memory hierarchy virtual memory secondary storage heterogeneous fabric multiprocessing cognitive neuromorphic instruction setarchitecturestypes cisc risc applicationspecific edge trips vliw epic misc oisc nisc zisc comparison addressing modes x86 arm mips power powerpc sparc itanium unicore microblaze riscv othersexecutioninstruction pipelining pipeline stall operand forwarding classic risc pipeline hazards data dependency structural control false sharing outoforder tomasulo algorithm reservation station reorder buffer register renaming speculative branch prediction memory dependence prediction parallelismlevel bit bitserial word instruction pipelining scalar superscalar task thread process data vector memory distributed multithreading temporal simultaneous hyperthreading speculative preemptive cooperative flynns taxonomy sisd simd swar simt misd mimd spmd processorperformance transistor count instructions per cycle ipc cycles per instruction cpi instructions per second ips floatingpoint operations per second flops transactions per second tps synaptic updates per second sups performance per watt ppw cache performance metrics computer performance by orders of magnitude types central processing unit cpu graphics processing unit gpu gpgpu vector barrel stream coprocessor asic fpga cpld multichip module mcm system in package sip by application microprocessor microcontroller mobile notebook ultralowvoltage asip systemson chip systemonchip soc multiprocessor mpsoc programmable psoc networkonchip noc hardwareaccelerators ai accelerator vision processing unit vpu physics processing unit ppu digital signal processor dsp tensor processing unit tpu secure cryptoprocessor network processor baseband processor word size 1bit 2bit 4bit 8bit 16bit 32bit 48bit 64bit 128bit 256bit 512bit others variable core count singlecore multicore manycore heterogeneous architecture components core cache cpu cache replacement policies coherence bus clock rate fifo functional units arithmetic logic unt alu address generation unit agu floatingpoint unit fpu memory management unit loadstore unit translation lookaside buffer tlb logic combinational sequential glue logic gate quantum array registers processor register register file memory buffer program counter stack control unit instruction unit data buffer write buffer microcode rom counter datapath multiplexer demultiplexer adder multiplier cpu binary decoder address decoder sum addressed decoder barrel shifter circuitry integrated circuit 3d mixed signal power management boolean digital analog quantum switch powermanagement pmu apm acpi dynamic frequency scaling dynamic voltage scaling clock gating performance per watt ppw related history of generalpurpose cpus microprocessor chronology processor design digital electronics hardware security module vteparallel computinggeneral distributed computing parallel computing massively parallel cloud computing highperformance computing multiprocessing manycore processor gpgpu computer network systolic array levels bit instruction thread task data memory loop pipeline multithreading temporal simultaneous smt speculative spmt preemptive cooperative clustered multithread cmt hardware scout theory pram model analysis of parallel algorithms amdahls law gustafsons law cost efficiency karpflatt metric slowdown speedup elements process thread fiber instruction window array data structure coordination multiprocessing memory coherency cache coherency cache invalidation barrier synchronization application checkpointing programming stream processing dataflow programming models implicit parallelism explicit parallelism concurrency nonblocking algorithm hardware flynns taxonomy sisd simd simt misd mimd dataflow architecture pipelined processor superscalar processor vector processor multiprocessor symmetric asymmetric memory shared distributed distributed shared uma numa coma massively parallel computer computer cluster grid computer hardware acceleration apis ateji px boostthread chapel charm cilk coarray fortran cuda dryad c amp global arrays mpi openmp opencl openhmpp openacc tpl plinq pvm posix threads raftlib upc tbb zpl problems deadlock livelock deterministic algorithm embarrassingly parallel parallel slowdown race condition software lockout scalability starvation category parallel computing media related to parallel computing at wikimedia commons 