The Nikon Expeed image/video processors (often styled EXPEED) are media processors for Nikon's digital cameras.
They perform a large number of tasks:
Bayer filtering demosaicing image sensor corrections/dark-frame subtraction image noise reduction image sharpening image scaling gamma correction image enhancement/Active D-Lighting colorspace conversion chroma subsampling framerate conversion lens distortion/chromatic aberration correction image compression/JPEG encoding video compression display/video interface driving digital image editing face detection audio processing/compression/encoding and computer data storage/data transmission.
Expeed's multi-processor system on a chip solution integrates an image processor in multi-core processor architecture, with each single processor-core able to compute many instructions/operations in parallel. Storage and display interfaces and other modules are added and a digital signal processor (DSP) increases the number of simultaneous computations. On-chip 32-bit microcontroller initiates and controls the operation and data transfers of all processors, modules, interfaces and can be seen as the main control unit of the camera.
In each generation Nikon uses different versions for its professional and consumer DSLRs / MILCs, whereas its compact cameras use completely different architectures. This is different from for example Canons DIGIC: its professional DSLRs double the processors of its consumer DSLR series. The Expeed is an application-specific integrated circuit (ASIC) built by Socionext specifically for Nikon designs according to Nikon specifications.
Technology The Nikon Expeed is based on the Socionext Milbeaut imaging processors with 16-bit per pixel multi-core FR-V processor architecture, using a highly parallel pipelined architecture which allows efficient hardware use, increasing throughput and reducing power consumption.
Each core uses an eight-way 256-bit very long instruction word (VLIW, MIMD) and is organized in a four-unit superscalar pipelined architecture (Integer (ALU)-, Floating-point- and two media-processor-units) giving a peak performance of up to 28 instructions per clock cycle and core. Due to the used four-way single instruction, multiple data (SIMD) vector processor units, data is processed with up to 112 data operations per cycle and core.
An on-chip 32-bit Fujitsu FR RISC micro-controller core is used to initiate and control all processors, modules and interfaces. The Expeed versions designated EI-14x and the Expeed 2 and 3 additionally include a HD video codec engine (FR-V based) and a 16-bit DSP with separate on-chip 4-block Harvard RAM which is usable for example for additional image- and audio-processing. The Expeed 3 (FR) (EI-158/175) is based on an improved Expeed 2 EI-154 with greatly increased processing speed.
A new architecture in the Expeed 3 (ARM) offers a highly increased speed in its image processor (with even two pipelines on the EI-160), its H.264 video encoder and is controlled by a dual-core ARM architecture microcontroller replacing the Fujitsu FR.
Image sensor interface CMOS/CCD image sensors are connected with additional external ICs, Nikon drivers, with the exception of the Nikon D3100. This is done by a mixed analog/digital interface which controls the sensor digitally, but receives analog signals with parallel 14-bit analog-to-digital (A/D) converters. The Expeed variants EI-142 and EI-158 use Nikon ASICs to connect all full-frame (FX) digital SLR sensors and additionally the Nikon D300/D300s with 12 simultaneous, parallel analog signal readout channels. Mainly due to a larger settling time it allows improved conversion accuracy compared to the four channels in the previous Nikon D2X / D2Xs, Nikon D200 or the Canon EOS 5D Mark II.Canon 5D Mark III vs 5D Mark II Vs NikonD800 VsNikon D700 six-channel readout is supported by the EI-149 and EI-154 used on the D90, D5000, D7000 and D5100.
The D3100 uses an Analog Devices sensor interface with integrated analog-to-digital converters. The result is a dynamic range only at the level of competitors like the (higher priced) Canon EOS 600D; lower than other Nikon DSLRs with the same Expeed 2 variant.
The Expeed EI-15x and EI-17x A/D converters allow an increased image sensor readout clock frequency with improved A/D converter accuracy, especially when using 14-bit sampling. Expeed A/D converters used for EI-149 or all EI-142 need considerably reduced clock rates (1.8 fps on Nikon D3X) for higher accuracy, limiting for example the D3s dynamic range at low ISOs.DXOmark Sensor ratings (needs Flash)
The Expeed 3 (ARM), first used in the Nikon 1 series, connects a data stream with 24 digital channels (bus), using A/D converters integrated on the image sensor chip.
Video processor The first variant, EI-14x originally included a video encoder capable of processing VGA resolution with 30 frames per second and MPEG-4 encoding. The software based video processor realized with FR-V processors enabled a reprogramming:Fujitsu Releases Three New "FR-V Family" Processors for Media Processing Fujitsu By using Motion JPEG encoding with 24p frame rate, Nikon achieved 720p HD video resolution. The advantages are easy JPEG image extraction, no motion compensation artifacts and low processing power enabling higher resolution, and the disadvantage is a larger file size, nearly reaching the 2 GB limit (for full compatibility) in 5 minutes. The Nikon D90 was the first DSLR with video recording capabilities.
The Expeed 2 (variant EI-154) greatly expanded the capabilities by its 1080p H.264/MPEG-4 AVC HD video encoder. It also offers an increased image sensor analog signal readout clock rate, reducing rolling shutter.
Compared to competitors from Canon (DIGIC, "IPP" compressionMPEG-2MPEG-2 and Video Compression Dr.Dobbs equivalent: Only one previous frame analyzed) the Expeed 2 offers video compression also based on complex B-frames (bi-directional differencing between frames and motion prediction), which has the advantage of higher quality (lower motion blur based on better motion compensation) even with significant higher compression ratio.Nikon D800/D800E: D-Movie shooting functions Nikon This compression requires considerably higher computing power.Video Compression Apple In 2012 the Canon 5D Mark III introduced a similar compression called "IPB".Understanding EOS HD Compression Options Canon Also introduced was "All-I", which uses the simpler I‑frames (coded pictures) without processing any differences between them, but using a low compression (high data rate) and behaves also in other uses like image extraction quite similar to Motion JPEG formerly used by Nikon.EOS-1D X: Video Shooting Workflow: New “ALL-I” video compression Cyberscholar
The Expeed 3 (FR) (variants EI-158 and EI-175) offers no significant change, but introduced the first DSLRs to offer uncompressed video output (8bit 4:2:2) over HDMI: Nikon D4, Nikon D800/D800E, Nikon D600, Nikon D7100 and Nikon D5200. The Expeed 3 (ARM) introduced high-speed video (slow-motion) in its enhanced H.264 HD video engine.
Variants Socionext specifies each Milbeaut generation with different numbers of processors. Nikon gives no details, but uses different designated processors in its professional and consumer lines. Although the Milbeaut (Expeed) is used in different Nikon designs and by other manufacturers, the software/firmware specifies many of its functions and details and the number of processors or included modules may vary in this ASIC.
Pre-Expeed
Early DSLRs DSLRs announced before August 2006 do not contain processors named Expeed (for example Nikon D70/D70s processor: EI-118), although that does not mean that these processors use a different architecture. Or the Nikon D200 processor (EI-126) uses the same, but greatly expanded firmware as the D80 (unofficial "Expeed").
Expeed
First used in the Nikon D3 and Nikon D300 in 2007, the Expeed was used later in the Nikon D3X, Nikon D700 and Nikon D300s, marked EI-142, and the consumer line variant with reduced processor cores in the Nikon D90 and Nikon D5000, marked EI-149. It is based on a Socionext Milbeaut imaging-processor with 720p Motion JPEG video encoder, DSP and FR-80 (EI-14x versions) core. It uses a 90 nanometer process technology.
Unofficial Expeed DSLRs The EI-137 variant in the Nikon D60 and Nikon D3000additionally found in the Nikon D40, Nikon D40x and Nikon D80is based on the older Milbeaut M-3 in 180 nanometer technology (like all former Expeed/Milbeaut since 2001). It includes a FR-71 core with only 12-bit, two-channel image sensor readout, no DSP, slower memory and has a reduced feature set.
Expeed 2 1080p H.264/MPEG-4 HD video encoder, improved face detection, image noise reduction and image distortion correction are the main improved features compared to EI-14x Expeeds. Used in the Nikon D7000, Nikon D3100 and Nikon D5100 and Nikon marked EI-154. Although image sensor readout clock frequency has increased by a factor of 1.75, A/D converter accuracy is improved, especially when using 14-bit. Image processor performance is increased, performing a higher continuous shooting frame rate even when high ISO noise reduction or Active D-Lighting is activated. The Expeed EI-15x are controlled by an integrated FR-80/FR-81 core. Power consumption is also decreased by the Socionext 65 nm process.
Expeed 2 (rebranded Expeed 1) The Nikon D3s processoralthough named Expeed 2uses nearly the same EI-142 processor as the previous D3/D3X featuring for example only 720p Motion JPEG video. It offers the same image sensor interface with identical speed and A/D converter accuracy, limiting the D3s dynamic range at 200 and especially 100 ISOs lower as the D7000/D5100.DXOmark: Nikon D3s: Comparison dynamic range (needs Flash)
Expeed C2 Variant used in some Coolpix compact cameras. Cheaper Nikon compact cameras use Sanyo or Zoran Coach image/video processors; both with a completely different technology and different firmware compared to the Expeed.
Expeed 3
Expeed 3 (FR) Compared to the previous Expeed 1 (EI-142), it offers the same improvements as the Expeed 2 EI-154 with Socionext 65 nm process, including increased A/D converter accuracy and image sensor analog signal readout clock rate, reducing rolling shutter. Computing power is highly increased. Marked EI-158, this variant is used by the Nikon D4 and Nikon D800. The EI-158 was the first Expeed to offer uncompressed video output (8bit 4:2:2) over HDMI.
The Nikon D600 (teardownNikon D600 Teardown ifixit), Nikon D3200 and Nikon D5200 (teardownInside the Nikon D5200 DSLR – Toshiba found! Chipworks) use an Expeed 3 (EI-175, differently marked ML-1131 on D5200), which is, according to Nikon, the same as used for the D4 and D800 series.D600 image quality: Image-processing engine EXPEED 3 Nikon.com Its architecture is close to the Expeed 2 variant EI-154 with some improvements like DDR3 memory, and with increased computing power. The D5200 uses a package on package with a 4 Gbit DDR3 SDRAM on top.
Expeed 3 (ARM) In the Nikon 1 series September 2011 Nikon introduced a new largely changed architecture – the main control unit it uses is an ARM microcontroller, which required new firmware compared to the totally different Fujitsu FR microcontroller used in all former Milbeaut and Expeed processors. It was also the first Expeed using digital image sensor readout – no analog image sensor interface was needed. In the Nikon 1 cameras introduced in September 2011, it used 1 GB fast DDR2 RAM packaged in 2x 4 Gbit chips. Nikon marked it EI-160, manufactured in the Socionext 65 nm process.