Instructions to use hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44") model = AutoModelForMultimodalLM.from_pretrained("hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44
- SGLang
How to use hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44 with Docker Model Runner:
docker model run hf.co/hector-gr/RLCR-Gemma-4-E2B-it-Math-seed44
RLCR Gemma 4 E2B-it — Math — seed 44
Gemma 4 E2B-it trained on Big-Math-Digits (mehuldamani/big-math-digits) to verbalize the confidence in its own answer with RLCR (our Gemma training of the RLCR baseline, not a checkpoint released by its authors).
Paper: Cross-Question Reliable Reinforcement Learning — Hector G. Rodriguez and Marcus Rohrbach.
Abstract
AI systems operating near the limits of their capabilities can maximize utility while minimizing the risk of error by abstaining or deferring questions they are not confident about to a human or a stronger model. Recently, large language models (LLMs) have been post-trained to estimate confidence in their own answers. To achieve this, methods reinforce confidence scores based on the correctness signal to each question-answer pair individually, with rewards that encourage assigning confidence 1 to correct answers and 0 to incorrect answers. However, estimating confidence scores is useful in that it enables selecting only the most confident answers, to maximize the share of solved questions while accommodating the user's risk tolerance. Assigning confidence 1 to correct answers and 0 to incorrect ones is theoretically optimal, but unattainable in practice and detrimental for selective prediction with a non-zero risk tolerance: confidences pushed to 0 and 1 carry no information about how answers compare to one another. We therefore introduce CARE (Cross-question Abstention Reward Estimation), which computes rewards over a set of question-answer pairs: At each of several risk tolerances, the confidences across the set induce an abstention threshold, and each confidence is rewarded for lying on the correct side of it. Since the threshold itself comes from the set, every verbalized confidence is reinforced relative to other questions' confidence, not against its own binary correctness label. On math, CARE increases the coverage at 5% risk (C@5) across MATH-500, GSM8K, and Big-Math-Digits from 0.8% to 42.4%, compared to existing pointwise correctness-matching rewards. When trained on multi-hop Wikipedia QA from HotpotQA, CARE improves C@5 in out-of-distribution general-knowledge, reasoning, and math benchmarks (including CommonsenseQA, GPQA, SimpleQA, and TriviaQA) from 12.3% to 23.6%.
Results of this checkpoint
Selective prediction, all values in %; C@r is coverage at risk r; AURC lower is better. ID: big-math-digits, math-500, gsm8k. OOD: commonsenseqa, gpqa, simpleqa, trivia, hotpotqa. The paper's Gemma table reports the mean of seeds 43 and 44.
| ID C@0 | ID C@1 | ID C@5 | ID C@10 | ID AURC ↓ | ID Acc | OOD C@0 | OOD C@1 | OOD C@5 | OOD C@10 | OOD AURC ↓ | OOD Acc |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.0 | 0.0 | 0.0 | 0.0 | 17.2 | 74.2 | 0.0 | 0.0 | 9.8 | 31.8 | 37.4 | 53.8 |
Use
Use the exact system prompt in system_prompt.txt, the saved chat template and all EOS IDs from generation_config.json. The paper evaluates with one completion, temperature 0 and a 4096-token generation limit. Outputs contain <answer>...</answer> and <confidence>...</confidence>. Rank answers by confidence and choose a threshold on suitable held-out data; a confidence score is not a guarantee of correctness.
These checkpoints require a Gemma 4-capable runtime. They are text-only post-training; the vision and audio encoders were frozen and multimodal use has not been evaluated. The training configuration is in training_config.yaml (per-device batch 4, gradient accumulation 96, maximum completion length 4096).
License and attribution
Base model: Google DeepMind Gemma 4 E2B-it, revision 70af34e20bd4b7a91f0de6b22675850c43922a03, Apache 2.0 (license and modification notice included). Post-training: Cross-Question Reliable Reinforcement Learning, Hector G. Rodriguez and Marcus Rohrbach. RLCR: Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (Damani et al., 2025).
- Downloads last month
- 36