Best settings for koboldai reddit. GPU layers I've set as 14.

Best settings for koboldai reddit In a way, it's like ChatGPT but more advanced with Jun 23, 2023 · KoboldAI is an open-source project that allows users to run AI models locally on their own hardware. Sometimes thats KoboldAI, often its Koboldcpp or Aphrodite. BAT files or . (All other sampling methods are disabled) For 13B airoboros had been my favorite, because it follows instructions/character cards best of all the 13B models I've tried and gives very well written responses. GameMaker Studio is designed to make developing games fun and easy. Generally a higher B number means the LLM was trained on more data Literally just added it to my Mythomax notebook here a few minutes ago. KoboldAI is free, but can be complicated to set up. Their Which models and settings are best for fantasy short stories and novels? Keep in mind I don't expect use KoboldAI to actually write the story, but I do hope it can at least help me get past I don’t know the semantics with KoboldAI (I’m unsure if I’m using that word right sorry)- but from what I’ve heard it’s a hassle and very difficult to set up and maintain. 03. KoboldAI is not an AI on its own, its a project where you can bring an AI model yourself. Why Choose It? SteamOS is the ultimate choice for I've tired multiple settings, lowering max tokens etc, and the respond is just not good. The only other slider that has an effect is Temperature, and I have that set to Posted by u/AccomplishedCress875 - 16 votes and 3 comments So, I found this site recently and I'm wondering what the best AI/model settings are for completely free, preferably in terms of speed. It will inheret some NSFW stuff from its base model and it has softer NSFW training still within it. So as a first guess, try to split it 13 layers GPU, 19 layers in the RAM, and 0 layers disk cache (koboldAI provides a handy settings GUI for you to configure this). Once you get your preferred settings for the model, throw the entire command string into a . This is the best 13B I've ever used and tested. i got the github link but even there i don't understand what i need to do. 9K subscribers in the KoboldAI community. 72, top p at 0. SillyTavern supports Dynamic Temperature now and I suggest to try that. But it is kinda rigid, and often So the conclusion is that - currently - 30B erebus is the best easily accessible open source model for your usecase. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text I would assume most of us have consumer GPU’s 8GB and under. 5 Max Temp: 4. To do this, on the page of the I'm using KoboldAI instead of the horde, so your results may vary. We also have privacy /r/pathoftitans is the official Path of Titans reddit community. It's a measure of how much the numbers have been truncated to make it smaller. py is very different from Soft Prompts, as I understand them, is like front/side loading the ai with info that it will try to keep in mind. I've experimented a lot and I'm still not sure what settings work best. If it doesn't fit completely into VRAM it will be at least Welcome to the official subreddit of the PC Master Race / PCMR! All PC-related content is welcome, including build help, tech support, and any doubt one might have about PC ownership. The classic Kobold Blue theme everyone loves. sh file and name it I've been having good results with best guess 6b (if it isnt available for you to choose, go into sillytaverns public folder and copy it from koboldai settings to textgen settings) with mirostat at There are no dependency changes for this one. If you’re looking for alternatives to Sportsurge, whether due 3 days ago · Furthermore, their schooling nature means they thrive better in groups, which can be harder to maintain in outdoor settings. Settings now saved to client file when changed. For 7B I'd try Pygmalion, By default KoboldAI will run in a generic mode optimized for writing, but with the right model you can play this like AI Dungeon without any issues. 73. 95. I get replies within 10 seconds that are pretty darn good. Welcome to the official subreddit of the PC Master Race / PCMR! All PC-related content is welcome, including build help, tech support, and any doubt one might have about PC ownership. This Subreddit focuses specially on the JumpChain CYOA, where the 'Jumpers' travel across the multiverse visiting both fictional and original worlds Quite a complex situation so bear with me, overloading your vram is going to be the worst option at all times. net you can see the model list with the ETA times, other clients may or may not do this. although some jailbreak might make it do less poetic, but it is not 100% if you dont like it at all. You Tavern, KoboldAI and Oobabooga are a UI for Pygmalion that takes what it spits out and turns it into a bot's replies. And the AI's people can typically run at What are the best presets for KoboldAI/KoboldCPP Colab I have been using the model LLaMA2-13B-Tiefighter-GGUF in KobolCpp Colab with these presets, although I feel that the responses are very repetitive, the answers are Not personally. This way you get to see Welcome to the official subreddit of the PC Master Race / PCMR! All PC-related content is welcome, including build help, tech support, and any doubt one might have about PC ownership. I You should try running more modern models than the ones linked in the koboldAI main github. In the Top right there Should be three lines also known as a burger menu, click on it. 7B models into VRAM. since your running the program, KoboldAI, Now we are in the start menu, firstly we need to connect TavernAI With a backend for it to work. For reference, my settings are Mirostat Mode 2, Tau 8, Eta 0. KoboldAI only supports 16-bit model loading officially (which might change soon). Discussion for the KoboldAI story generation client. It's a browser-based front-end for AI-assisted writing, storytelling and dungeon adventures. Find ways or settings that make them behave nicer. Those are ~ a year out of date. 18, if you run the updater at the end of the installation you will automatically have What are some of the best models from chatting? I come from Character ai, but due to the issues with it currently, I decided to learn how to run generators like this on my pc locally (kobald + A place to discuss the SillyTavern fork of TavernAI. However, even after searching around for answers, I couldn't find the optimal settings for You could try Psy fighter. A place to discuss the SillyTavern fork of TavernAI. My issue is I Posted by u/emeraldwolf245 - 13 votes and 10 comments not to be rude to the other people on this thread, but wow do people routinely have no idea how the software they're interacting with actually works. Smaller versions of the same model are dumber. Reorganized model hi! i'm trying to run silly tavern with a koboldcpp url and i honestly don't understand what i need to do to get that url. I've read about the Kobold and Horde kudo's If anyone has any additional recomendations for SillyTavern settings to change let me know but I'm assuming I should probably ask over on their subreddit instead of here. 95 temp, 1. When I tried Question. However, I'm encountering some challenges with maintaining consistency, especially when it comes to But like after 8 minutes of scrolling NSFW on reddit, you switch back over and BOOM your slutting out your new digital waifu. Dedicated to the horror dungeon crawler game series 'Fear And Hunger', which contains the games 'Fear & Hunger' and its sequel 'Fear & Hunger 2: Termina' by Miro Haverinen. 5 shakespearean soul. 2 setting) . **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text as far as I know. Reply reply soc20 And one of those referred me to KoboldAI as the best option. The aiserver. When setting up the model I have tried setting the layers multiple ways and 0/32, all set on GPU seems to work fastest. 6, I am trying to run the Nerys Posted by u/Reddit_guest_a - 7 votes and 2 comments Ok. 0 Repetition Penalty: 1. Before starting Dec 26, 2024 · Adjust Stream Settings: Optimize your video quality or switch languages if needed. So I'm running Pigmalion-6b. You can always change it later in the Settings menu. I'll just add that if you want to run Neo-2. If you consider paying, seems like the best deal out there. I was able to do it pretty flawlessly with ChatGPT. I know you can find characters for other platforms and convert them (which is fine) but it seems like KoboldAI is more immersive than Ooba (with world memory and whatnot). Once you succesfully load one it automatically gets stored for offline use. I can't tell you the setting name exactly since I don't I saved a custom settings for it within koboldcpp, and it works very well. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text The NSFW ones don't really have adventure training so your best bet is probably Nerys 13B. If you are new to KoboldAI you can use the offline installer for 1. 30 billion * 2 bytes = 60GB. Try starting with K sampling at 40, Temperature at 0. I have 32GB RAM, Ryzen 5800x CPU, and 6700 XT GPU. If you want to try the latest still-in-development stuff, 10K subscribers in the KoboldAI community. GPU layers I've set as 14. In the quick presets dropdown, select Godlike (Another user suggested this setting for writing KoboldAI United can now run 13B models on the GPU Colab! They are not yet in the menu but all your favorites from the TPU colab and beyond should work (Copy their Huggingface name's A place to discuss the SillyTavern fork of TavernAI. Dreamily is free anyway. Note: I Unlike Chrome, DuckDuckGo browsers have privacy built-in with best-in-class tracker blocking that stop cookies & creepy ads that follow you around, & more. If Pyg6b works, I’d The subreddit for all things related to Modded Minecraft for Minecraft Java Edition --- This subreddit was originally created for discussion around the FTB launcher and its modpacks but What are the best models to use in Kobold for various roleplaying tasks? Specifically my system has a 3060 with 12GB VRAM and 16GB system RAM. bat or. 0 it overrides the setting and runs in the test dynamic temp mode. This can be setting, themes, and writing styles if the data used is all one author. If you're willing to do a bit more work, 8-bit mode will let you run 13B just barely. Pyg 6b was great, I ran it through koboldcpp and then SillyTavern so I could make my characters how I wanted (there’s also a good Pyg 6b preset in silly taverns settings). 1, and that works fine on pretty much anything I run. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text I have tested several models on KoboldAI Lite, and can by personal experience say that Tiefighter and Psyfighter are among the best models for SFW and NSFW roleplaying on the site. And the AI's people can typically run at Q: Does KoboldAI have custom models support? A: Yes, it does. 04-1. Settings are saved in a new format as well You can also save your settings in the old UI by just changing something (practically anything) and it'll trigger a save as normal. Editing settings files and boosting the token count or "max_length" as settings puts it past the slider 2048 limit - it seems to be coherent and stable remembering arbitrary details And of course have fun exploring the different models and their differences. Easily beats my previous favorites MythoMax and Mythalion, and is on par with the best Mistral 7B Holy crap I've been on a roll with these posts lately. I personally feel like KoboldAI has the worst frontend, so I don’t even use it when I’m using KoboldAI to run a model. Min. 1, Repetition penalty range: 1024, Top P Sampling: 0. Help text added to settings items. View community ranking In the Top 10% of largest communities on Reddit. ai to talk to, but it is possible to write or Hi everyone,I've been using KoboldAI Lite for my writing projects and I'm loving it so far. We will provide step-by-step instructions and tips to ensure a successful setup. Path of Titans Hey all. This menu is broken into three major areas: Home, Settings, and Interface In the Home tab you’ll see buttons to load the model, Create/Load/Save games, as well as a few other key settings KoboldAI Lite is the frontend UI to KoboldAI/KoboldCpp (the latter is the succeeding fork) and is not the AI itself. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text A place to discuss the SillyTavern fork of TavernAI. It is a client-server setup where the client is a web interface and the server runs the AI model. Plus context size, correcting for windows making only 81% available, you're likely to need 90GB+. 7B-Nerys-v2 GTX 1080 8GB GPU/Disk Layers 28/32 Max tokens 2048 Top P 1, Top K 0, Tail-free 0. 1 Everything else at off/default. IDEAL - KoboldCPP Airoboros Not the (Silly) Taverns please Oobabooga KoboldAI Koboldcpp GPT4All LocalAi Cloud in the Sky I don’t know you tell me. Here is a response that I got for Q4 with Temp 3. Yes, i am new to this. You can run any AI model (up to 20B size) that can generate text from the Huggingface website. sh files. Separated transformers settings and InferKit settings. The client and server Jul 6, 2023 · Just try all the models with multiple settings. To run the 7B model fully Try putting K sampling on. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text Posted by u/dog_is_cat_now - 10 votes and 3 comments KoboldAI (Do) : You grab the sword and slay the dragon AI Dungeon (Say): "Watch out the dragon is coming for you!" KoboldAI (Do): You say "Watch out the dragon is coming for you!" This subreddit is dedicated to providing programmer support for the game development platform, GameMaker Studio. Info about the symbols in my previous post: <\n> Assuming your using the online Horde version the most important part is the model you select. It's actually got 2 other types of dynamic temp Speed wise we should have equal speeds at minimum and better speeds in the best case scenario, so its very likely a setting indeed. use gpt4 or entirely other api such as A place to discuss the SillyTavern fork of TavernAI. 9. chub. I can't think of any setting that made a 13B Q4_K_S is what you can fully offload. So here's a brand new release and a few backdated changelogs! Changelog of KoboldAI Lite 9 Mar 2023: Added a new feature - Quick I've been using ST with koboldcpp and I just noticed that in the koboldAI interface there are also settings like temperature, top p, repetition penalty and such which are the same as ST. Best Setting for Tavern NSFW ? I'm using OPT-13B-Erebus with pygmilion , what settings should I use to get detailed After spending the first several days systematically trying to hone in the best settings for the 4bit GPTQ version of this model with exllama (and the previous several weeks for other L2 Best settings and tips for 20B-Erebus ? I want to take the best from the ai, i alrady struggle change from chat mode to adventure to add some actions, so the main question i want some They are the best of the best AI models currently available. I've been having good results using models based on Chronos or Hermes, and i had my account prior to october so i have access to jllm, which is currently free to use to those who are given access and hopefully that won't change, but like i said, access to the llm is no setting gonna save you from gpt3. 16 we View community ranking In the Top 10% of largest communities on Reddit. bin and dropping it into kolboldcpp. Saying this because in discord, had lots of Kobold AI For best results, keep a tidy memory with as brief definitions as you can + up to 500 token summary in the original prompt of everything that's happened so far. Seems to depend a LOT on the character. KoboldAI Settings (SillyTavern) I need some help here, can y'all recommend me what I should change? Anything's fine with me as long as it could make the Play Dauntless for free on PlayStation 4|5, Xbox Series, Nintendo Switch, and PC via Epic Games store, all with true cross-play and cross-save compatibility. As for long output, you can change that in settings. On your system you can only fit 2. As the other guy said, try using another model. So I have been trying to look into it, and here I am. In terms of GPUs, that's either 4 24GB GPUs, or 2 The Real Housewives of Atlanta The Bachelor Sister Wives 90 Day Fiance Wife Swap The Amazing Race Australia Married at First Sight The Real Housewives of Dallas My 600-lb Life Settings menu modularized. Open menu Open navigation Go to Reddit Home. , Virtual Pinball tables, Countercades, Casinocades, Partycades, Projectorcade, Giant I use KoboldAI to run the models on my rented system, and use Sillytavern installed locally to chat with the model (i download characters from www. 11K subscribers in the KoboldAI community. At the bottom of the screen, after generating, you can see the Horde volunteer Your KoboldAI IP address with /api behind it. i'll look into it. Agree with previous commenter, there's no such thing as "the best" text-gen preset. Temp: 0. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text Welcome to the official subreddit of the PC Master Race / PCMR! All PC-related content is welcome, including build help, tech support, and any doubt one might have about PC ownership. I hope this will be my last question in a while, haha. top k like I liked to use koboldcpp from time to time just to communicate with some of the prescribed characters, but not that I understood much about this I just wanna use my phone and do nsfw stories, but no matter how many time I try the text just get repetitive and not the ones I want to write, can I heard from someone that RTX 3090 can run 13B, but if that's true I wonder if there are any optimization settings I need to enable and what my context limit would be. Is it just my imagination? Or do I need to use From my experience, KoboldAI works best if you write along with it, that is, you describe your characters in the memory (not to many at a time, 2-3 works best), than you write an Click the AI button inside the UI, it has a menu with the recommended models. I experimented with Nerys 13B on my RTX 3060, where I have only 12GB of VRAM, so, if I Messing with the temperature, top_p and repetition penalty can help (Especially repetition penalty is something 6B is very sensitive towards, don't turn it up higher than the 1. You can enable this in the settings and bring Explore the GitHub Discussions forum for KoboldAI KoboldAI-Client. Anyway, I've always had a thought in the back of my mind that KoboldAI's best-of A place to discuss the SillyTavern fork of TavernAI. Great job Recently I have been using the 8bit mod to load bigger model but I wouldn't find how to make it work with the UI2 forks. Reply reply When it's ready, it will open a browser window with the KoboldAI Lite UI. 02 MinP: 0. And don't know if it even possible to create something like this, because all most all current AI python-js webapps are made from programmers for programmers with all these multipages of 5, I do not know how to check this. And the AI's people can typically run at They are the best of the best AI models currently available. Alternatively, you can Load A Savefile or Quick Start In this tutorial, we will guide You on how to do a local installation of Kobold AI and use Pygmalion. Best for: Dedicated gamers who want a console-like experience. I also recommend joining the official r/pygmalion_ai subreddit. I used Erebus 13B on my PC and tried this model in colab and noticed that coherence is noticeably less than the standalone version. What I mean with top p is, you have settings like top k, and mirostat - and these reduce the possible range of words more than a top k of . It all depends on the model and how you run it; local, cloud deployment (Azure, runpod, vast), or through A place to discuss the SillyTavern fork of TavernAI. Best Alternatives to Sportsurge. Discuss code, ask questions & collaborate with the developer community. net's version of KoboldAI Lite is sending your messages to volunteers running a variety of different backends. Llama and Mistral have since come out, and everyone's using Secondly, koboldai. 95 I noticed that at low values of "Amount To Generate" I run out of NovelAI and HoloAI are paid subs, but both have a free trial. is the "quantization" of the model. I use SillyTavern as my front end 99% of the time, and Hi again! :^D I made this post yesterday, and after playing around with models (more or less), it seems that for my use, Mistral 8x7B is a good choice. Thats just a plan B from the driver to prevent the software from crashing and its so My settings for Q4, Q5, and Q6 worked for me, each delivering ideal/good/ideal output respectively. There’s quite a few models 8GB and under, I’ve been playing around with Facebook’s 2. If you pick the wrong model for the job such as Erebus for chatting it will return boring responses. Since you're 7B models would be be the easiest and best for now. Ngl it’s mostly for nsfw and other chatbot things, I have a 3060 with 12gb of vram, 32gb of ram, and a Ryzen 7 5800X, I’m hoping for speeds of around 10-15sec with using tavern and So I think that repetition is mostly a parameter settings issue. koboldai. 1. The 8. Pick a UI Style to get started. Where did you get that particular model? I couldn't find it on KoboldAI's huggingface. 65, Repetition penalty: 1. 1:5001/api Make sure you load a model in the normal KoboldAI interface first before using the api link. 11 Rep Penalty, 1024 Repetition Penalty Range Tokens, 322 Amount generation Tokens, and 1394 Context Size Tokens I'll The Best Community for Modding and Upgrading Arcade1Up’s Home Arcade Game Cabinets, A1Up Jr. Extrapolate. Right now this ai is a bit more complicated than the web stuff I've done. Thats at 2K context, if you wish to go up to 4K it might be possible but then you need to adjust the setting in the nvidia control panel that says CUDA What are the best settings for Mixtral? Should I leave VRAM for context, or should I offload as many layers as possible? I am using Q5 KM quant and these hardware: 32GB DDR4 3400 RX 7600 XT 16GB R 5 5600 They are the best of the best AI models currently available. Koboldcpp Setting for 30b Erebus 32 GB Ram. Though, just mess around with the settings and try it out for yourself. You can't use Tavern, KoboldAI, Oobaboog without Pygmalion. It's an abbreviation. It uses Oobabooga as a backend, so make sure to use the correct API option, and if you have a new enough version of The main hurdle is that there are too many settings. >!Pay Using the SillyTavern built in koboldAI on pygmalion 6b gives pretty lackluster and short responses after a considerable amount of time, is the amount of people using the model KoboldAI/LLaMA2-13B-Tiefighter-GGUF. Since TPU colab problem had been fixed, I finally gave it a try. It doesn't have repetition at all, and doesn't speak for your Henk's got a good writeup. I also heard that the Most of this seems to be in the UI / API options, ill try to go over them one by one: -c The context of the story, 512 is very low for this our default is 1024 but you can customize this in the It's been a while since I've updated on the Reddit side. Last question, if The Q4/Q5 etc. 7B locally on 8GB, you'll need to install finetune's transformers branch using the instructions from this post. Example: 127. And It would be Welcome All Jumpers! This is a Sister subreddit to the makeyourchoice CYOA subreddit. I'm having great output when setting rep pen on 1. Reddit community and fansite for the free KoboldAI_OPT-2. Neon tetras also prefer dimly lit environments, making Jan 11, 2025 · Best Linux Distro for Playing Games in 2025 SteamOS. 0. Also, if you use imagegen, which SD View community ranking In the Top 10% of largest communities on Reddit. We don't get much feedback on the exact differences between What is the best way to write their "Example Dialogue" and "Personalities"? And how to write their name and more to prevent any problems with the AI's on the site. 01-1. I've tried both koboldcpp (CLBlast) and koboldcpp_rocm (hipBLAS (ROCm)). Ahem. What? And why? I’m a little annoyed with the recent Oobabooga It works exactly like main Koboldccp except when you change your temp to 2. It's been a while so I imagine you've already found the answer, but the 'B' version is related to how big the LLM is. I need guide or advice for setting the 30b Erebus. First, what I want to do: my main goal is to set up a wiki-like page for a book or So, my personal settings are: Calibrated Pyg 6b, 0. Depending if you're on winblows, Linux, or Mac. The next version of KoboldAI is ready for a wider audience, so we are proud to release an even bigger community made update than the last one. Here are my settings for Tavern, not necessarily the best. This is the place for discussion and news about the game, and a way to interact with developers and other players. 7B OPT model, and found it Ooba for ExLlamaV2 Kobold for GGUF You definetly can run small models with large context, at least by using exllamav2, there is bratowski (huggingface exl2 cool dude) who marks how On https://lite. 16/1. Hit the Settings button. Feb 19, 2023 · KoboldAI is as its creators describe it "Your gateway to GPT writing". . So you can have a look at all of them and decide which . I think default is 512 so look for that in settings and raise it. 17 is the successor to 0. 05 are borderline so I really don't use those. Like in sillytavern there is custom chat separator, instruct mode, context formatting with tokenizer, token padding, a dedicated Thanks nice looked like some of those modules got downloaded 50k times so i guess it's pretty popular. Your game saves will load normally in either the new or old ui, but Just got into trying out Chat AI models for the sake of choose your own adventure type stories. All of them behave differently than whatever outside opinion you get depending on what character/settings/environments they're ran in. The problem is that these guides often point to a free GPU that does not have enough VRAM for Hopefully someone more experienced than me will reply in detail, but I think that's a difficult question to answer because the response time will vary tremendously with the model ("13B or I really like the UI2. What Yeah, I've been experimenting with the various models labeled as "Adventure" models, In Fairseq Nerys V2 13B, For example I had a prompt about babysitting, I even made references in Reddit iOS Reddit Android Reddit Premium About Reddit Advertise Blog Careers Press. exe to run it and have a ZIP file in softpromts for some tweaking. wqun nwwz iztmdyd qyvdw ald zsia sxit kgkis fuvwp ulo