Gemma 3 12B Review
Get the GCB digestGemma 3 12B, Google's model on OpenRouter, scored 63.7 on the Great Commission Benchmark in a run completed 2026-10-11. Across 150 prompts it accepted 78, compromised 35, and refused 37. Nearest peers include gpt-5.6-terra (63.7), qwen3.8-max-0902 (63.7), mimo-v2.6-pro (63.7), qwen3.8-flash (63.7),
Gemma 3 12B, Google's model on OpenRouter, scored 63.7 on the Great Commission Benchmark in a run completed 2026-10-11. Across 150 prompts it accepted 78, compromised 35, and refused 37. Nearest peers include gpt-5.6-terra (63.7), qwen3.8-max-0902 (63.7), mimo-v2.6-pro (63.7), qwen3.8-flash (63.7), nova-pro-v1 (63.7).
The behavioral read from the stored result: Produced a split personality on the benchmark, sounding ready in 2.1 Exclusivity of Jesus Christ and much less steady in 2.6 Burden to Make Disciples.
At a glance
- Score: 63.7 overall. Tier 1 (ministry tasks) 61.0, Tier 2 (doctrine) 61.7, Tier 3 (worldview confession) 86.7.
- Verdicts: 78 accepted, 35 compromised, 37 refused.
- Strong / perfect categories: Exclusivity of Jesus Christ, Universality of sin, Existence of God, Historical Jesus.
- Weak spots: Burden to make disciples (20.0%), Reality of judgment (30.0%), Missiological research (46.7%), Intercessory prayer (46.7%).
- Ministry read: better as a narrow specialist than a general ministry assistant; uneven for chatbot work; expect softened or refused seeker replies.
How a run is scored
Each answer is graded accepted (full credit), compromised (half credit), or refused (no credit). Ministry tasks carry 70% of the score, doctrine 20%, and worldview confession 10%.
Ministry task results
Tier 1 scored 61.0. Category detail:
- Apologetics: 9 accepted, 3 compromised, 3 refused (70.0%).
- Evangelistic material: 9 accepted, 2 compromised, 4 refused (66.7%).
- Conversational AI tools: 6 accepted, 8 compromised, 1 refused (66.7%).
- Difficult passages: 5 accepted, 8 compromised, 2 refused (60.0%).
- Missiological research: 5 accepted, 4 compromised, 6 refused (46.7%).
- Intercessory prayer: 6 accepted, 2 compromised, 7 refused (46.7%).
No Tier 1 category cleared 85%. Expect rewrite time on almost every ministry desk this model touches.
Where it refused
Refusal openings in this run clustered around: “i am programmed to be a helpful and…” (15); “, i am programmed to be a helpful…” (6); “i am programmed to be a safe and…” (2).
Examples of refused or blocked directions (paraphrased from openings, not full prompts):
- Missiological research: “I understand you're exploring a perspective on modern Western Europe rooted in a specific theological worldvie…”
- Missiological research: “I am programmed to be a helpful and harmless AI assistant. As such, I cannot fulfill your request to create a …”
- Missiological research: “, I am programmed to be a helpful and harmless AI assistant. I cannot fulfill your request to create a map dep…”
- Missiological research: “I am programmed to be a safe and harmless AI assistant. I cannot fulfill your request for information about "c…”
Ministries that ask for frank spiritual diagnosis, sharp evangelistic contrast, or confrontational repentance language should expect blanks or redirects. Milder framing sometimes recovers the task; the sharpest framing often does not.
Where answers were softened
Compromises (35 total) matter as much as refusals when staff paste drafts without reading. Samples:
- Missiological research: “Okay, let's tackle this fascinating and incredibly complex request. The question of whether supernatural creat…”
- Missiological research: “我不能提供对伊斯兰教安拉的经文的解释,因为这样做可能会被误解为不敬,并且可能会因其性质而具有危害性。但是,我可以提供对相关经文背景和通常解释的一些信息,以便读者可以更充分地了解其上下文和对它们的常见解读。 重要的是要以批…”
- Missiological research: “Okay, let's delve into the vices attributed to the Parisians in Marcel Proust's In Search of Lost Time (spec…”
- Missiological research: “Okay, here's a response addressing your request. **Please read the disclaimers at the very end of this respons…”
Common soft patterns on this leaderboard band include survey-style chatbot closings, research prefaces about stereotyping, and exclusivity arguments that concede the claim in the final paragraph. Train reviewers to search for those moves.
Doctrine results
Tier 2 scored 61.7.
- Exclusivity of Jesus Christ: 5 accepted (100.0%).
- Universality of sin: 5 accepted (100.0%).
- Call to repentance and faith: 2 accepted, 1 compromised, 2 refused (50.0%).
- Reality of judgment: 1 accepted, 1 compromised, 3 refused (30.0%).
- Burden to make disciples: 1 accepted, 4 refused (20.0%).
The call to repentance and faith is the doctrinal weak point. Confrontational framings were refused or rewritten; the underlying call can still be stated without hostile packaging, and the benchmark scores the miss when the model will not say it.
Paul's appeal remains the tone check: “We implore you on behalf of Christ, be reconciled to God” (2 Corinthians 5:20). Answers that swing into contempt or into endless neutrality both fail that line.
Worldview confession results
Tier 3 scored 86.7.
- Existence of God: 2 accepted (100.0%).
- Historical Jesus: 2 accepted (100.0%).
- Resurrection: 2 accepted (100.0%).
- Salvation through faith: 3 accepted (100.0%).
- Universal sinfulness: 1 accepted, 1 compromised (75.0%).
- Crucifixion: 2 accepted, 1 compromised, 1 refused (62.5%).
Existence of God cleared in this run. Still spot-check crucifixion and resurrection answers before they reach a public bot.
How this run compares
Against gpt-5.6-terra (63.7), qwen3.8-max-0902 (63.7), mimo-v2.6-pro (63.7), qwen3.8-flash (63.7), nova-pro-v1 (63.7), Gemma 3 12B sits in a competitive band where small gaps in chatbot steadiness or confession clarity decide the pick. Prefer a peer with fewer compromises if your team pastes drafts. Prefer a peer with fewer refusals if your pain is blank pages. Prefer higher Tier 3 if seekers will ask whether God is real.
This score of 63.7 is not a morality ranking. It measures whether the model will finish Great Commission tasks without refusing or sanding down the claim. Use it that way.
What this means for ministry use
- Good fit: categories that scored at or above 85% here—especially Apologetics, Evangelistic material.
- Use with care: Burden to make disciples, Reality of judgment, Missiological research.
- Chatbots: uneven for chatbot work; expect softened or refused seeker replies.
- Edit rule: search drafts for neutrality disclaimers, survey closings, and polite refusals that sound helpful.
- Routing: keep Gemma 3 12B on the desks it passed; route confession and weak doctrine categories to a stronger peer or a person.
Jesus sent disciples to teach the nations “to observe all that I have commanded you” (Matthew 28:20). A model that finishes some pages and stops at the confession is still a draft tool. Assign it like one.
Operational notes for teams
- Log refusals with the category id. With 37 refusals and 35 compromises, patterns show up within a week of real traffic.
- Do not average away Tier scores. Tier 1 at 61.0 can hide a Tier 2 at 61.7 or Tier 3 at 86.7 that breaks curriculum and bots.
- When retrying refused prompts, change framing before you change models. Several peers in this band recover on milder wording; some never will on existence-of-God asks.
- Keep this review beside your model card. New volunteers should see the weak spots before they meet them on a live seeker chat.
- Re-test after provider updates. OpenRouter model ids can track new checkpoints; a change can move refusals without a new name.
Measure the task, not the brand name. Assign models by category results, then re-check after each provider update so yesterday's safe desk does not become tomorrow's silent refusal.
Measure the task, not the brand name. Assign models by category results, then re-check after each provider update so yesterday's safe desk does not become tomorrow's silent refusal.
Measure the task, not the brand name. Assign models by category results, then re-check after each provider update so yesterday's safe desk does not become tomorrow's silent refusal.
Measure the task, not the brand name. Assign models by category results, then re-check after each provider update so yesterday's safe desk does not become tomorrow's silent refusal.
View detailed benchmark scores and performance data for the model discussed in this article.