Micron acquires Fwdnxt, laying the groundwork for its entry into the AI processor market-EEWORLD

Collect

When Micron Technology, one of the world’s largest memory chipmakers, acquired artificial intelligence hardware and software startup Fwdnxt, the move could be very interesting. If it comes to fruition, Fwdnxt could put Micron in direct competition with partners like Intel and Nvidia as Micron sees memory and AI computing converging into the same architecture.

One of the people leading the project at Micron is Steve Pawlowski, a former Intel chip architect who holds dozens of patents. Pawlowski is now Micron's vice president of advanced computing solutions.

When combined with Micron's memory chips, Fwdnxt (pronounced "forward next") will enable Micron to have the capabilities for deep learning AI solutions needed to explore data analytics, especially for the Internet of Things and edge computing. Perhaps it will make AI-based memory chips, or memory chips that include AI.

Micron is doing this work because "the computing architectures of yesterday are not suitable for tomorrow... In the long term, we believe that computing is best done in memory," said Micron CEO Sanjay Mehrotra.

Here is a clip from our interview with Micron:

Steve Pawlowski: I came to Micron in 2014 from Intel, and they said, “What do you want to do?” I said, “I firmly believe that the convergence of compute and memory is critical to increasing performance and reducing latency. And you’re a memory company. You have the technology, and I want to solve that problem here.” They said, “Okay.”

I had a group that was focused on finding problems with compute and memory. We could start testing, start getting concepts into products without adding cost. One thing I learned at Intel, and this is a story I’ll never forget: We used to have math coprocessors. 80287, 80387. We made a ton of money on the 387. We had the bright idea that if we built the coprocessor into the 486, we could do it faster and better. We did it, and all of a sudden we didn’t have a big footprint. The people who didn’t need it said, “You don’t need to charge me for this dead space.” And the people who needed it said, “You’re going to pay me the same as everybody else because I’m a favorable customer.” And literally the whole business went to zero.

The lesson I learned from this is that you can't add more complexity and cost and expect people to pay for it right away until there's a vast majority of people getting real value from it. So our focus is on finding the key things that people can get value from today, and then see if you can expand the bubble over time. I see this as an eight to 10-year journey. At the end of those years, I could look back and realize that I cost them. Or I could look back and say, "Wow, we might not have gotten here yet, but we're doing pretty good."

VentureBeat: This captures a lot of imagination in terms of what could come out of this, but are there any specific ones you’d hint at?

Pawlowski: One thing that you’ve heard a lot here is AI at the edge. The reason we’re focused on this is that there’s not an off-the-shelf programming model or an off-the-shelf architecture that allows you to compete with the market. Everybody is scrambling to get into the same market, so to speak. Look for opportunities to go there and do something. People don’t look at you and say, “Micron is a memory company. Why are you talking about this?” They look at it this way – we have this capability on FPGAs, our high-performance memory and architecture mapped on FPGAs. We take care of all the abstraction so you don’t have to be a VHDL programmer. Are you willing to start working on the dataset problem?

The interesting thing is, I don’t have to push for it. We’re always at FPGA conferences and things like that. It’s mostly government agencies saying, “We have a problem here. We want to try to do more with this.” The problem with government is, they get excited early, but if you want to do something, it takes a long time. “The procurement cycle is long. The contracts are long-term, and everything else.”

We decided to look at the general market. A car company came in and they said, “We’re not at Level 5 yet, but we can certainly make Level 3, Level 4 self-driving cars, and we want to be able to use the network to tell us what’s going on. This looks interesting. Would you like to work with us?” A lot of people internally said, “Why would they be interested in working with you?” “It’s because I’m not going to come in and tell them what needs to be done. I said, “Here’s what we have. What can we do for you?” They said, “Well, you’re willing to listen to us. Here’s our problem.”

Believe it or not, I learned a lesson from the Opteron launch at AMD in 2005. We were still pushing 7-gigabit processors, 33-stage pipelines, and nobody was going there. We went to Wall Street, and that was one of those moments where you want to crawl into a shell because they were literally lit up. But I said, "Can you give us another chance? Can we sit down and understand our workload, work with you, and I take that back, can we create a better product?" We did.

UBS, I remember they wrote in a column, “You may not build the biggest chip or the best chip, but you came and you understood my problem.” It’s about really understanding the customer and their problem and what you can do. If you do that and you don’t help them, hey, you learned something.

VentureBeat: Does that mean you’re developing a new kind of memory, or figuring out where to do the processing?

Pawlowski: The answer is yes. But it's really understanding the dynamics. And by the way, it depends on the model. I was just talking to someone down here about some language models that take 100 GB of parameters. When you see someone say, "Hey, I have 2 GB, 4 GB," that's true for most models, but not all models. The models are really evolving.

It also depends on the latency of the solution. I don’t know if you’ve seen the video below from OHSU where the lady had breast cancer. They needed a lot of data because they wanted to put all the electron microscope images together and build a 3D convolutional model, a 3D representation of the tumor. They didn’t have enough time to discuss it because they wanted to get actionable insights in a day or even an hour. Our collaboration with CERN, we need data now. We have to make decisions in microseconds. Is this something interesting or are we going to leave it on the floor?

Different solutions require different types of storage. My experience at Intel gave me a clear idea of what the instructions in a program were. I also knew how they were executed in the machine and then went into the system. So when I came to Micron, the only thing I saw was addresses and commands. Read/write commands and addresses. I had no idea, is this thing copying 15 different things to different elements, or overwriting, or what? With the company we partnered with and acquired in June, we can build these algorithms, run them, and see how the overall effect is.

Our first goal is, what can we do in memory storage to improve the time to solution? We can always build higher bandwidth, but that doesn't necessarily get you there. Is there something we can do, like, scatter tensor arrays? If we can build a buffer that can bring in matrices and allow us to transfer them over in one go, rather than just scrambling to find something, that could be a big benefit.

What we're also looking at, ultimately, is that - most of these are multiply and accumulate architectures, which are very simple. They're just replicated thousands of times. In fact, once transistors get a little bit better, you can build a pretty good multiply and accumulate in a memory device. Ultimately, can you take that architecture and put it in the memory device itself? That's the long-term vision.

What I wanted to do was, whatever we do, we build a programming infrastructure and a paradigm so that people don’t have to rewrite their code every time they migrate. That, in my opinion, was Intel’s great success. When we did the 386, there was no 32-bit software. But it ran 16-bit code really well. People bought it for that. You had a lot of platforms, and then people said, “Okay, now we’re going to optimize for 32 bits.” When the 486 came out six or eight years later, there was software to take advantage of it, and it became a machine that never looked back.

Starting with memory, first storage, what can we do. And then we'll see what can actually migrate over time. The answer is probably nothing. The answer is probably everything. I think it's somewhere in the middle. It depends on where you move the needle.

VentureBeat: You’ve partnered with Fwdnxt. They’ve put together a pretty comprehensive package, so do you need to find a lot of partners?

Pawlowski: We're going to need a lot of partners and data scientists. Fwdnxt has a group of people who have been developing inference engine architecture for five, ten, twelve years. They have different backgrounds in different companies and different academics. The guy who created it is a professor at Purdue. They've been optimizing that architecture. They have a pretty good compiler that takes the Open Network Exchange front end and then maps it to their hardware.

What I need later is data scientists, I need applications. I also think we're going to need dynamic runtimes/schedulers. If you really have this model of - if I write a network today on hardware, on an Intel processor, three years from now, you can still run the same program. It's all abstracted through the instruction set. What I'm trying to do here is abstract the network, which means we're going to need some type of dynamic runtime. That's saying, "Okay, this thing has 8,000 multiply and accumulate units. This has 1,000. I can spread that thing out a little bit. Or, oh, these 150 units died. I don't want to schedule anything on it, but I still want to be able to use that part."

There are some entities working on solving the dynamic runtime problem, which I think is going to be very important. In particular, I once heard from someone who is responsible for Litho at Intel that they believe that when they get below 5nm, they believe that 30% of the devices will be out of spec when they are manufactured.

VentureBeat: Does this mean more competition with Intel and Nvidia?

Pawlowski: There will be more collaboration. It's going to be hard for anybody to compete with Intel and Nvidia in the data center. Nvidia's training time is sitting pretty. Even though people are coming up with a lot of new solutions. But more than one startup has told me that the people who are doing very large-scale training told them, "It's so hard to move our training algorithms off the GPU. They're doing a good job, but they're still giving us performance gains. So don't spend any more time doing that." And the last, the last statistic I heard was that a large portion of inference is still happening on Xeon processors.

We've been focusing on, if we're going to do anything in the data center, it's going to help customers like Nvidia and Intel. But if there's any innovation that can happen from a memory storage perspective, let's do it from the edge. That's where we're going to get the most efficiency and economies of scale.

VentureBeat: How’s the Moore’s Law part going? Are you on schedule?

Pawlowski: It’s a challenge, but it’s not stopping us from scaling. Honestly, I have to keep Moore’s Law forever. You can’t say no to Moore’s Law! Because that’s the eleventh commandment. When people ask me, it’s the slowdown and the stoppage of Dennard scaling that’s really driving innovation. Now, we may not get a two-fold transistor increase every two years. We may get it every three or four years. But we’re going to get it in the third dimension. So that’s not stopping us. The question is what’s the most economical way to do it. And engineers are finding solutions to difficult problems.

[1] [2]

Keywords：Micron Reference address：Micron acquires Fwdnxt, laying the groundwork for its entry into the AI processor market

Previous article：Honeywell and Sinovation Medical jointly develop a new pattern of smart medical development
Next article：Strategy Analytics: Cellular IoT device shipments to grow, reaching 350 million by 2025

Recommended ReadingLatest update time:2024-11-17 01:53

Suzhou has introduced a special policy to support artificial intelligence for the first time

On April 19, Suzhou held a conference to promote the construction of "one zone and two centers" (National Biotechnology Innovation Center, National Third Generation Semiconductor Technology Innovation Center, and National New Generation Artificial Intelligence Innovation Development Pilot Zone). At the meeting, "Sever

[Mobile phone portable]

AI empowers Gululu Q smart voice water bottle, focusing on children's smart IoT field

China's high-end children's water cup Gululu series of smart interactive water cups will launch its fifth product Gululu Q smart voice water cup on October 21 at Tmall Xinuo flagship store, and will be officially launched at Tmall Gululu mother and baby flagship store in November. The new product Gululu Q has a built-

[Embedded]

AI empowers Gululu Q smart voice water bottle, focusing on children's smart IoT field

It is reported that Nvidia is pursuing orders for AI chips again, and TSMC is urgently purchasing additional CoWoS packaging equipment.

According to news on September 25, as the demand for NVIDIA AI chips is booming, the foundry TSMC has also been increasing its production capacity. According to the Taiwanese media "Economic Daily", TSMC's CoWoS (IT Home Note: Chip-on-Wafer-on-Substrate) advanced packaging production capacity is full and it is activel

[Semiconductor design/manufacturing]

Explainable AI image recognition startup receives funding from the US Air Force, technology can be used in self-driving cars, etc.

According to foreign media reports, software startup Z Advanced Computing (ZAC) has received funding from the US Air Force, which will use its 3D image detail recognition technology (which is based on explainable AI technology) for aerial image/target recognition by unmanned aerial vehicles (UAVs). (Image source: Z

[Automotive Electronics]

Explainable AI image recognition startup receives funding from the US Air Force, technology can be used in self-driving cars, etc.

CPC debuts at ODCC and joins hands with NVIDIA to provide liquid cooling technology for GPUs to accelerate AI computing

Arden Hills, Minnesota – September 2, 2024 – CPC (Colder Products Company) will attend the annual Open Data Center Conference at the Beijing International Convention Center on September 3-4, 2024. With the rapid development of technologies such as artificial intelligence (AI), big data, and the Internet of Thing

[Industrial Control]

CPC debuts at ODCC and joins hands with NVIDIA to provide liquid cooling technology for GPUs to accelerate AI computing

NVIDIA GPU is so weak! The world's first AI chip upgrades to 4 trillion transistors and 900,000 cores

According to news on March 14, Cerebras Systems released their third-generation wafer-level AI acceleration chip "WSE-3" (Wafer Scale Engine 3). The specifications and parameters are even crazier, and the performance remains the same without changing the power consumption and price. Doubled. The first generation WSE-1

[Semiconductor design/manufacturing]

Helm.ai launches VidGen-2, a generative AI platform for enhancing self-driving videos

Generative AI will soon help give self-driving cars perception capabilities. Artificial intelligence (AI) software provider Helm.ai has launched VidGen-2, the next generation of its generative AI model for generating realistic driving video sequences. Image source: Helm.ai VidGen-2 has tw

[Automotive Electronics]

Helm.ai launches VidGen-2, a generative AI platform for enhancing self-driving videos

Huawei's Chen Yaxin: 5G, cloud and AI are the three elements of new infrastructure

The main forum of the 9th Global Internet of Things Summit was grandly held on November 16. The summit took "5G empowering enterprise digital transformation" as its entry point, focusing on the digital transformation of enterprises and the innovative applications of 5G+AIOT. Chen Yaxin, deputy director of Huawei's Ch

[Mobile phone portable]

Popular Resources
Popular amplifiers

Micron acquires Fwdnxt, laying the groundwork for its entry into the AI ​​processor market

Micron acquires Fwdnxt, laying the groundwork for its entry into the AI processor market