← All topics

Explore local inference

Related products and practices, with creator experiences across different uses.

With use and recommendations from

Jeffrey Morgan Alex Finn
+1 more
Greg Isenberg

DGX Spark

Product

2 creators

Running 20B to 120B+ parameter models locally on a desktop; chaining multiple units to run 400B models

"We were one of the first people, along with Elon and a few others, to receive the DGX Spark as well, which sits on your desk and provides 128 GB of unified memory to run that kind of 20 to 120B model range. You can absolutely stack them on your desk almost ..."

Jeffrey Morgan

Open Models Change The Economics of AI ▶ 25:19
More source quotes

Running local AI models for unlimited free inference

"The DGX Sparks a really great plug-and-play device. You just plug it into the wall and it works. It has 128 gigabytes of unified memory which basically means you can run any model that goes up to 128 gigs."

Alex Finn

Hermes Agent Desktop: Full Setup + Real Use Cases ▶ 38:07
View all product sources →

Ollama

Software

1 creator

Running local AI models via command line with a local API that apps can connect to

"Ollama is a little more builder oriented or developer oriented. You install it, you run a command like 'ollama run gemma4' and now you have a model running locally with an API your apps could talk to."

Greg Isenberg

I'm Obsessed With Local AI. Here's Why ▶ 5:39
More source quotes

Running AI models locally for privacy, cost control, latency, and understanding local vs. cloud architecture

"If you could run these models locally with something like Ollama, LM Studio, you start to understand what can happen on your own machine and what needs the cloud and what needs to touch private docs and what should stay behind the wall."

Greg Isenberg

"Learn AI” Is Bad Advice. Learn This Instead ▶ 2:32

Running local AI models via command line

"There's two main names to know, Ollama and LM Studio. Ollama is usually the favorite of a lot of my developer friends because it runs from the command line."

Greg Isenberg

Claude Fable 5 is BANNED. What to do? ▶ 8:01
View all product sources →
About these recommendations

Products are related to this topic through catalog tags. Quotes describe the creator’s stated use; they are not necessarily recommendations for this topic.

Creator counts cover all uses and all dates. Each person counts once per product family. Paid promotions and passing mentions are excluded. Items are ordered by creator count, then name.

Search all local inference matches →