Hi there, I want to share some thoughts and want to hear your opinions on it.
Recently, AI developments are booming also in the sense of game development. E.g. NVIDIA ACE which would bring the possibility of NPCs which run an AI model to communicate with players. Also, there are developments on an alternative to ray tracing where lighting, shadows and reflections are generated using AI which would need less performance and has similar visual aesthetics as ray tracing.
So it seems like raster performance is already at a pretty decent level. And graphic card manufacturers are already putting increasingly AI processors on the graphics card.
In my eyes, the next logical step would be to separate the work of the graphics card, which would be rasterisation and ray tracing, from AI. Resulting in maybe a new kind of PCIe card, an AI accelerator, which would feature a processor optimized for parallel processing and high data throughput.
This would allow developers to run more advanced AI models on the consumer’s pc. For compatibility, they could e.g. offer a cloud based subscription system.
So what are your thoughts on this?
Your GPU is an AI accelerator already. Running trained AI models is not as resource demanding as training one. Unless local training becomes universal, AI acclerators for consumers make very few sense.
Apple’s been putting AI accelerators in phones for years. They use it for things like real time face recognition to help the camera decide where to focus.
The GPU can do the same task, but AFAIK it uses something like 20x more power.
I think it totally makes sense to have hardware accelerated AI even if only to free up the GPU for other tasks.
The newest gen GPUs have sections dedicated to AI already, so we effectively already have dedicated AI accelerators.
Yes there are but the op is talking about discrete AI accelerators…
Unless the AI processing is much more specialized than graphics, I think manufacturers would put that effort into making more powerful GPUs that can also be used for AI tasks.
They would try to alleviate the cost on running GPU by making an AI accelerator chip like Tensor Core, but it’ll get bottleneck by limited VRAM when Neural Net models require steep amount of memory. it’s more productive to have something like NPU that runs either on RAM or by it’s own memory chips offering higher amount of capacity to run such neural net and avoid the roundtrip data copying between GPU and CPU.
We saw this happen a long time ago with PPUs. Physics Processing Units. They came around for a couple of years, then the graphics cards manufacturers integrated the PPU into the GPU and destroyed any market for PPUs.
It was before my time but… If physX cards are any indication, then no.
The PhysX debate was also before my time. But I read into it, and it seems like they solved it partly software based. Please correct me if I’m wrong, I just skimmed over the PPU subject. But with AI we are talking about hardware limitations, especially memory.
Currently, AI operations mean a lot of time-consuming copy tasks between CPU and GPU.
Good question, but I’d say that the same train of thought went through dedicated physics cards. I’d guess that an AI card should have a great value proposition to be worth buying.
For compatibility, they could e.g. offer a cloud based subscription system.
I’m not sure where you’re going with this, but it feels wrong. I’m not buying a hardware part that cannot function without a constant internet connection or regular payment.
I’d guess that an AI card should have a great value proposition to be worth buying.
Sure the card should have great value or must have an accessible price. It probably also depends on how “heavy” the tasks get. But seeing e.g. OpenAI struggling with requests, it may be useful to decentralize the processing (with running the model locally on the user’s pc).
I’m not sure where you’re going with this, but it feels wrong. I’m not buying a hardware part that cannot function without a constant internet connection or regular payment.
Maybe this statement was a bit confusing. What I meant was, that in a transition phase developers could choose to allow the usage of a dedicated accelerator card to run everything locally and offline. And for people who don’t have or want such a card they could provide a cloud based subscription model, where the processing is done on remote servers.
Yeah, that makes more sense.
By the way, Microsoft just announced that they are trying to do the opposite, and move users to thin clients with Windows itself being a cloud service.
That said much less processing power is required to run a model than to train it. Games also would not require big models, since they only need to know the game lore, not all the world’s knowledge.
There are limited LLMs out there that can run on a phone.
Absolutely, I would suggest looking into two separate devices that focuses solely on AI acceleration:
and
Two very interesting articles. Thank you for that!
Especially the analog processor is a game changer with having the computation directly in memory. Generally, analog computers are a very interesting subject!
The Apple silicon (https://en.wikipedia.org/wiki/Apple_M1#Other_features and M2 and their variants) have 16+ neural engine cores for on chip AI, separate from the GPU cores. But it’s still a package deal.
I could see them splitting it out for cases of high end AI clusters and dedicated servers for that use case, but I feel like their current goal is to make sure that those cores are included in common hardware so that everyone can leverage local AI and not worry about “does this person have hardware to do this?” Issues.
I think current industry thinking is that making those cores commonplace helps the adoption of AI for everyday software more so that requiring a separate add-on card.
Restate your question without any of the following buzzwords: A.I., artifical intelligence, machine learning.
Now, clearly describe what you are talking about.
IMO, it looks like you’re completely lost in the sauce.
Look into what Mystic AI was doing. It’s effectively what you were talking about but based in reality :)
What is the significance of semiconductors in chatbot technology?How do semiconductors enhance chatbot capabilities?Can chatbots powered by semiconductors understand and respond to human emotions?What role do semiconductors play in voice-based interactions with chatbots?
Tips: the datasheet (https://www.icdrex.com/the-future-of-communication-chatbots-powered-by-semiconductors/) may help you a little.
Future cpu’s will probably feature ai accelerators (like igpus) but I don’t think local ai models will be demanding enough to require a separate card. There’s just a huge gap between the computation power needed for rudimentary ai tasks (which can run on a simple accelerator) and doing the same things more accurately (which might require more power than a gpu). I don’t think the diminishing returns would justify having separate cards for most people and games when they can just run a simpler model that’s almost as good on the cpu accelerator.