For inferencing it is “capable of delivering up to 157Top/s for multiple concurrent AI pipelines, combined with high-speed IO interface options including 10Gbit/s Ethernet, 5G and Wi-Fi”, according to Supermicro.
Dubbed ARS-E103-JONX, it measures : 185 x 80 x 140mm and works over -25 to +60°C with 0.7m/s airflow.
Application processing in the comes from 64bit Arm Cortex-A78AE (v8.2) cores, eight in the 16G version of Orin NX and six in 8G version.
ARS-E103-JONX-H2 has the eight core CPU, which means that it ‘Ampere’ GPU has 1,024 CUDA cores and 32 Tensor cores, running at 918MHz.
On top of this are two 614MHz 20Top/s (sparse INT8) deep learning accelerators (DLAs).
ARS-E103-JONX-L2 is the six core 8Gbyte version of the computer, which has a slower (765MHz) GPU and only one (610MHz) DLA.
Supermicro’s claim of 157Top/s comes from the 16Gbyte processor in Nvidia’s 8V 40W ‘Maxn_super’ mode, which up-clocks the GPU to 1,173MHz, the DLA to 1,229MHz. It is quoted in sparse INT8 Top/s, and the equivalent dense INT8 figure is 78Top/s.
Can the computer sustain 157Top/s?
“ARS-JONX-E103 is capable of reaching the 157Tops using the ‘Maxn_super’ setting, and with the full complement of the interfaces on the motherboard, at 60°C. Supermicro temperature specification are always with 100% CPU performance, no degradation, no throttling,” the company told Electronics Weekly, adding the caveat: “When the POE supply, 5G and Wi-Fi is needed, the maximum temperature will be 50°C.”
It also cautioned: “Please keep in mind that the 157Top/s is using sparse int8 under benchmark conditions. In real life, the performance will depend on additional factors – mainly model quantising and optimisation.”
In one of its 5V lower-power modes, Nvidia rates the Jetson Orin NX at up to 100 sparse INT8 Top/s and 50 dense INT8 Top/s for the version with 16Gbyte of on-board EEC LPDDR5 memory (’16G’), or up to 70 sparse and 35 dense INT8 Top/s for the 8Gbyte (‘8G’) version.
For main memory there is a M.2 PCIe 4.0 (x4) NVMe slot (M-key 2280 default).
Interfaces include 1x 10Gbit/s Ethernet, 4x 1Gbit/s Ethernet, 1x USB 3.2 Gen2 Type-C, 3x USB 3.2 Gen2 Type-A, HDMI for a display, 2x RS232/422/485 serial, 1x CAN, 4x digital inputs and 4x digital outputs.
Options include a DIN rail clip for mounting.
Find the ARS-E103-JONX-H2 on this Supermicro web page, and find more about the Orin NX’s power modes in this Nvidia web page
Announced very recently, Nexcom picked Nvidia’s Jetson Orin Nano for an edge AI PC for smart agriculture
