People David Idea Part of our work in the Kumubot cluster involves being able to work on both text as well as audio recordings - the idea was the figure out the capabilities of Qwen3-Omni on our Blackwell hardware Details * Blackwell software support (at least for the RTX 6000 Pro)
People David Idea Joe wanted to know what the capabilities of https://github.com/HKUDS/AI-Trader look like in the context for local LLMs Details * Looks like the assumption of the repo is that you can test multiple AI in agentic fashion against historical stock market data over a period
People: David Idea: To setup our Kumubot cluster, I went down a rabbit hole benchmarking different quantization methods for Qwen3 8B and 32B models to see which ones actually deliver the best accuracy-to-speed tradeoff in real-world use Details: • ExLlamaV3-4bpw surprisingly beat even BF16 on LiveBench accuracy (60.0 vs 58.
People: David Idea: Based on conversations with Keao about machine-translation benchmarking, I ran a bunch of LLM models through MMLU-ProX (lite) French benchmark tests (biology section + full 14-topic suite) to see which ones actually deliver the best mix of speed and accuracy in French. This started because we hypothesized that
People * David Idea Exploring memory-efficient fine-tuning techniques for improving Hawaiian-to-English translation using Apple's MLX framework, comparing multiple approaches and optimizing for Mac hardware. Details * Successfully fine-tuned gemma-3-4b-it-4bit on Mac M1 Ultra (128GB RAM) achieving 0.8296 semantic similarity score, a 3.6% improvement over the base model * Discovered
People: * David Idea: * Comparing fine-tuning performance on MacBook M3 Max, Mac Studio M1 Ultra, and Nvidia 4090 using MLX and Unsloth Details: * Tested fine-tuning Phi-3-mini-4k-instruct model * Followed this Jan 2025 MLX guide for Apple hardware * Used Unsloth library for Nvidia GPU * Dataset had 627 examples and used 500 training steps
People David Idea Exploring cloud GPU performance, cost, and usability for running open-source AI models (in comparison to local hardware and in context of recent learning on software to handle concurrent users) Details * Compared RunPod serverless and pods using Nvidia (vLLM) and AMD (sglang) * Benchmarked Nvidia 3090, 4090, RTX 6000
People: * David Idea: * Comparing two open-source projects, Onyx and Morphik, for self-hosted semantic search (and deep research) across Google Drive files (PDFs, Docs, Sheets, and Slides) Details: * Onyx: Great text-based search, flexible embedding models * Morphik: Powerful multi-modal search (text, images, graphs) * Onyx easily connects and syncs with Google Drive * Morphik
A Real-World Test of Prompt-to-UI AI Creating a usable interface for a brand-new app often eats up early project time. To see how far AI can shorten that stage, we pointed Magicpatterns.com at a brand idea called Po‘owai, a cash-flow tool for non-profits that have to juggle restricted
Bringing the Ho‘omo‘o App Home — No Tech Jargon Required You played with Ho‘omo‘o in Bolt.ai’s browser editor and now you’d like to run it on your own computer and save it on GitHub for safekeeping. Good news: it’s basically a three-minute job.
* People * David * Idea * For a given set of local LLM models that a set of users (or community) wants to use - how do you measure performance/UX on ways to serve it (based on ease of use, speed of processing and generation, level of accuracy, and features like multi-user-context
* People * David * Idea * For a given set of local LLM models that a set of users (or community) wants to use - what is the best/easiest way to serve it (based on ease of use, speed of processing and generation, and features like multi-context and * Details * Engines like llama.