Do you use it to help with schoolwork / work? Maybe to help you code projects, or to help teach you how to do something?
What are your preferred models and why?
You must log in or register to comment.
I’ve been experimenting with it for different use cases:
- Standard chat style interface with open-webui. I use it to ask things that people would normally ask ChatGPT. Researching things, vacation plans, etc. I take it all with a grain of salt and also still use search engines
- Parts of different software projects I have using ollama-python. For example, I tried using it to auto summarize transaction data
- Home Assistant voice assistants for my own voice activated smart home
- Trying out code completion using TabbyML
I only have a GeForce 1080 Ti in it, so some projects are a bit slow and I don’t have the biggest models, but what really matters is the self-satisfaction I get by not using somebody else’s model, or that’s what I try to tell myself while I’m waiting for responses.
As a voice assistant server for my home assistant setup.
Edit. I leaned it by watching a Networkchuck video on YouTube. Basically a Piper integration for “voice to text” then a Whisper intergration for “text to voice”, and openWakeWord integration for wake word.
That sounds interesting! Can you describe what software you used for that? And how powerful does the hardware has to be?
I employ this technique to embellish my email communications, thereby enhancing their perceived authenticity and relatability. Admittedly, I am not particularly adept at articulating my thoughts in comprehensive, well-structured sentences. I tend to favor a more primal, straightforward cognitive style—what one might colloquially refer to as a “meat-and-potatoes” or “caveman” approach to thinking. Ha.
I’m in the process of trying them out again
Phi4 has been okay for me, and I use deepseek R1 32B quantized for some coding tasks. Both are a lot for my aging m1 MacBook Pro to handle.
Lately Ive been trying deepseek 8b for document summaries and it’s pretty fast but janky.
What I’m working towards is setting up an RSS app and feeding that into a local model (freshRSS I think lets you subscribe to a combined feed) to build a newspaper of my news subscriptions, but that’s not viable until I get a computer to run as a server.
I’m currently using it to generate initial contact emails, and generate contextual responses to received replies, for a phishing project at work.
in order to prevent phishing, right? (cue anakin/padme meme)
lol. Uhhhhhhh not so much lol.
I work on an internal red team, so covert in prod operations instead of limited scope one off pen tests. We actively phish employees, but any victim user isn’t named in the report and we provide follow up training with them that’s not shame based and it’s with the operators directly, not some mandatory online class annual training bullshit.
It sucks, but this is a huge vector of initial compromise for APTs, and I work in an industry and for a company that are both extremely frequently targeted by APTs, so we have to do what we do. It lets us identify gaps in security and signature known ttps so our defensive teams can id those alerts when they pop.
i like the addition of ‘not shame based’ haha
Bro it’s a huge problem, companies will blame the employee that opened the malicious document instead of blaming their fucking abysmal internal security controls, detection, and response.
at work, right?
It is at work i promise lol.
I currently don’t. But I am ollama-curious. I would like to feed it a bunch of technical manuals and then be able to ask it to recite specs or procedures (with optional links to it’s source info for sanity checking). Is this where I need to be looking/learning?
you might want to look into RAG and ‘long-term memory’ concepts. I’ve been playing around with creating a self-hosted LLM that has long-term memory (using pre-trained models), which is essentially the same thing as you’re describing. Also - GPU matters. I’m using an RTX 4070 and it’s noticeably slower than something like in-browser chatgpt, but I know 4070 is kinda pricey so many home users might have earlier/slower gpu’s.
deleted by creator
How have you been making those models? I have a 4070 and doing it locally has been a dependency hellscape, I’ve been tempted to rent cloud GPU time just to save the hassle.
I’m downloading pre-trained models. I had a bunch of dependency issues getting text-generation-webui to work and honestly I probably installed some useless crap in the process, but I did get it to work. LM Studio is much simpler, but less customization(or I just don’t know how to use it all in lm studio). But yea, I’m just downloading pre-trained models and running them in these UI’s (right now I just loaded up ‘deepseek-r1-distill-qwen-7b’ in LM Studio). I also have the nvidia app installed and I make sure my gpu drivers are always up to date.
