Skip Navigation
technology @hexbear.net git [he/him, comrade/them] @hexbear.net

I watched Nvidia's Computex 2024 keynote and it made my blood run cold

www.techradar.com I watched Nvidia's Computex 2024 keynote and it made my blood run cold

Nvidia's pre-Computex keynote address was certainly something, and none of it felt good.

I watched Nvidia's Computex 2024 keynote and it made my blood run cold
12
12 comments

  • I don't think Nvidia CEO Jensen Huang is a bad guy, or that he has nefarious plans for Nvidia, but the most consequential villains in history are rarely evil. They just go down a terribly wrong path, and end up leaving totally forseeable, but ultimately inevitable ruin in their wake.

    forgive the ruling class for it knows not what it does, always misguided always redeemable never responsible

  • Good. I would rather see the entire amazon burn than give up my ability to AI prompt myself as a nordic gamer chad and the person I'm arguing with as an antisemitic crying soyjak.

  • I'm going to slap down a hot take that is somewhat tangential to the article. I think deep learning models of many kinds are actually good. I use ML for speech to text, and text to speech. I use home assistant, with it's AI powered tools. Hell, I've even generated images to help me visualize descriptions of things (it was not useful tbh). All of these uses help me, both with my disability and with just making my day easier, and it all ran LOCAL HARDWARE. The fact is a almost any of the use cases for ML can be made small enough to run locally, but there are almost no good local ML accelerators. Something like 90% of sentences sent to Google home was requests to set a timer. You don't need a data center for that.

    This is the true intent of these ridiculously power hungry data center cards, to build out a sort of rentier class of hardware owners (Nvidia the largest among them) that others have to rent from to stay cutting edge. This has been the dream since software as a service was thrown around 10 years ago. But just like streaming services, SaaS, and indeed rent, once the demand has been fully monopolized or trusted into whatever sectors it may end up in the price will skyrocket. ML could have been a real revolution in improving people's lives, but instead the hardware makers saw only a way to develop a route to make money with no work of their own, and the theft of work from others.

    • All of these uses help me, both with my disability and with just making my day easier, and it all ran LOCAL HARDWARE. The fact is a almost any of the use cases for ML can be made small enough to run locally, but there are almost no good local ML accelerators. Something like 90% of sentences sent to Google home was requests to set a timer. You don't need a data center for that.

      I run LibreTranslate on matapacos.dog for inline post translation (and at home to avoid Google knowing every scrap of foreign language text I read) and it is a similar story. It runs locally (doesn't even require a GPU) and makes no remote requests. Language models developed for specific purposes can accomplish great things for accessibility with much lower performance requirements than the gimmicky shit Silicon Valley tries to pawn off as "artificial general intelligence."

      • Exactly! PCs today are powerful enough to run them in decent time without acceleration too, it would just be more efficient to have it, ultimately saving time and energy. I would be interested in seeing how much processing power is wasted to calculate what are effectively edge cases in a models real work load. What percentage of GPT-4 queries could not be answered accurately by GPT-3 or a local LLaMA model? I'm willing to bet it's less than 10%. Terawatt-hours and hundreds of gallons of water to run a model that, for 90% of users, could be ran locally.

    • How do you feel about stuff like google coral?

      • I remember hearing about coral when it first came out, and it is exactly what I want to see more of. Low-cost, efficient AI accelerators for home use. I haven't personally used it but I have used Nvidia's Jetson Nano, which is similar but more power hungry. Coral does has some issues though, namely that you are locked into tensorflow lite (a similar issue with the Jetson series, but less bad since you can translate models into tensorRT), and the edge TPU hasn't had an update since 2018 as far as I can tell, and surely there have been breakthroughs to make them more efficient, able handle larger and newer models, or at least cheaper since then. I could be conspiratorial and suggest that any more powerful and they will cut into google cloud service money, but it's moot.

        In short, I like it and I want to see more of it. In a capitalist view, I want more competition in this space.

        Edit: I know that multiple edge TPUs can be used together, but larger solutions for municipal or business use would be nice as well.

  • I'm really excited to see the shovel sellers get owned when stonks-down after everybody realizes them thar hills were always filled with pyrite.

You've viewed 12 comments.