The Uncensored Gemma 4 12B Is Taking Off on 12 GB GPUs

Share

Gemma 4 12B Heretic is a community model that uses abliteration to strip out the refusal behavior of Google's Gemma 4 12B, and it runs on a 12 GB graphics card in GGUF form. It has been downloaded more than 100,000 times in the past 30 days, but Google did not release it and the repo publishes no benchmark data. This guide covers what it is, which files fit which hardware, which claims hold up, the risks, and a prompt that lets your AI agent help you pick a local model.

Machine translation. Read the original

A community model on Hugging Face called Gemma 4 12B Heretic has been spreading fast lately. It is built on Google's instruction-tuned Gemma 4 12B. Its creator used "abliteration" to strip out the model's tendency to refuse requests, then packed it into GGUF files that run on a 12 GB graphics card. It has been downloaded more than 100,000 times in the past 30 days. However, Google did not release it, and the model page itself says the model is "completely unaligned" and that whoever uses it is responsible for its output. Before you install it, make sure you understand what it is, what hardware it needs, and where the risks are.

What it is

  • Based on Gemma 4 12B: Google's open model, which can understand images. The 12-billion-parameter size is fairly friendly to ordinary graphics cards.
  • What "abliteration" means: Researchers found that a model's refusal behavior roughly corresponds to one specific direction inside it. Removing that direction from the weights makes the model refuse far less often. This version was abliterated with Heretic, an open-source tool.
  • Who published it: A community user called culturerevolt, not Google and not the author of the tool. The repository went live in June, and it only recently started spreading on social media.
  • License: The repository lists Apache 2.0. It is a derivative of Gemma, so whether Google's Gemma terms still apply is decided by Google's official terms. A license tag from the community is not official permission.

Which files run on which machines

File Size VRAM recommended by the repo
IQ3_XS 5.27 GB 8 to 12 GB, thin-and-light laptops and integrated graphics
IQ4_XS 6.64 GB 12 to 16 GB
Q4_K_M 7.38 GB 16 GB
Q5_K_M 8.55 GB 16 GB or more
Q6_K 9.79 GB 20 to 24 GB
Q8_0 12.67 GB 24 GB

To let it see images, you also need an extra 0.18 GB vision file (mmproj). The files can be loaded straight into local tools like Ollama and LM Studio, and they work without an internet connection. On a 12 GB graphics card, IQ4_XS is the safer choice. Q4_K_M is a bit tight, and you also need to leave room for long context.

Which of the claims hold up

  • "Code, reasoning, and multimodal skills are all intact": The repository only explains how it tried to preserve reasoning during compression. It does not publish any benchmark results. Abliteration itself may also make the model worse at other things, and there is no public data on that yet.
  • "Official GGUF repository": Not true. This is a community repository.
  • "Verified by 100,000 people": Download counts are not verification. They only show that many people downloaded it.

Where the risks are

Once refusals are removed, the model will answer any request, and it is also more likely to follow instructions hidden in web pages or documents.

  • Be careful when you connect it to an AI agent: Agents read web pages and call tools, so a model that never refuses is easier to steer with instructions hidden in that content. Actions such as deleting files or sending messages should only go to the official model.
  • If you offer it to others, you are responsible for filtering: The output is not filtered. If you put it in a product that other people use, the responsibility is yours.

Let your AI agent help you choose

If you only want to run a local model on your own computer, paste this prompt into your AI agent so it can help you pick the right version:

I want to run a Gemma 4 12B class local model offline on my own computer, for [e.g. drafting writing / organizing notes / local Q&A].

1. Check my GPU model and VRAM in read-only mode (on a Mac, check unified memory), and tell me which quantized version fits (IQ4_XS, Q4_K_M, etc.), along with how much context headroom to leave.
2. Recommend the official Gemma 4 12B from Google and its trusted GGUF versions first, and tell me where to download them and whether Ollama or LM Studio is easier.
3. Explain the difference between the community "uncensored" versions and the official version, and what extra risks they add when used inside an AI agent.
4. For now, only give me a plan. Do not download or install anything until I confirm.

Notes

  • Official terms are what count: Gemma's terms of use include a list of prohibited uses. Removing refusals does not change those restrictions.
  • Check the source before you download: Community models can be changed or taken down at any time, so check the publisher and file notes before you download anything.

References

Done with this one? Mark it to track your path.Sign in
Save it for laterSign in
Questions about this piece?Ask AI about this

Comments (0)

No comments yet. Start the conversation.

Sign in to join the discussion.Sign in