============================= test session starts =============================
test_BibleRAG_refactored.
py::test_bible_data_loaded Loading SentenceTransformer on CPU…
Loading CrossEncoder on CPU: BAAI/bge-reranker-v2-m3
Loading cached Bible embeddings…
FAISS index built with 31098 vectors.
✓ Bible data loaded correctly – PASSED
Note saved successfully at 2026-06-03 19:58:19
Notes Index built with 5 personal reflections.
✓ Notes indexing successful – PASSED
✓ Bible retrieval works – PASSED
✓ Notes retrieval works – PASSED
✓ Context expansion works – PASSED
Using prompt template for: gemma
✓ Prompt template loaded for model_type: gemma – PASSED
✓ Notes formatting for LLM works – PASSED
✓ Retrieval latency: 4.06s – PASSED
Test_end_to_end_query:
Loading LLM from E:/Python/AI/gemma-2-9b-it-Q5_K_L.gguf…
Here is the Bible context:
** Main verse: Isaiah 32:17 **
[Isaiah 32]
14: For the palace will be forsaken. The populous city will be deserted. The hill and the watchtower will be for dens forever, a delight for wild donkeys, a pasture of flocks,
15: until the Spirit is poured on us from on high, and the wilderness becomes a fruitful field, and the fruitful field is considered a forest.
16: Then justice will dwell in the wilderness; and righteousness will remain in the fruitful field.
17: The work of righteousness will be peace, and the effect of righteousness, quietness and confidence forever.
18: My people will live in a peaceful habitation, in safe dwellings, and in quiet resting places,
19: though hail flattens the forest, and the city is leveled completely.
20: Blessed are you who sow beside all waters, who send out the feet of the ox and the donkey.
—–
** Main verse: Matthew 10:13 **
[Matthew 10]
10: Take no bag for your journey, neither two coats, nor sandals, nor staff: for the laborer is worthy of his food.
11: Into whatever city or village you enter, find out who in it is worthy, and stay there until you go on.
12: As you enter into the household, greet it.
13: If the household is worthy, let your peace come on it, but if it isn’t worthy, let your peace return to you.
14: Whoever doesn’t receive you or hear your words, as you go out of that house or that city, shake the dust off your feet.
15: Most certainly I tell you, it will be more tolerable for the land of Sodom and Gomorrah in the day of judgment than for that city.
16: “Behold, I send you out as sheep among wolves. Therefore be wise as serpents and harmless as doves.
—–
** Main verse: Hebrews 3:8 **
[Hebrews 3]
5: Moses indeed was faithful in all his house as a servant, for a testimony of those things which were afterward to be spoken,
6: but Christ is faithful as a Son over his house. We are his house, if we hold fast our confidence and the glorying of our hope firm to the end.
7: Therefore, even as the Holy Spirit says, “Today if you will hear his voice,
8: don’t harden your hearts as in the rebellion, in the day of the trial in the wilderness,
9: where your fathers tested me and tried me, and saw my deeds for forty years.
10: Therefore I was displeased with that generation, and said, ‘They always err in their heart, but they didn’t know my ways.’
11: As I swore in my wrath, ‘They will not enter into my rest.’”
—–
** Main verse: James 3:18 **
[James 3]
15: This wisdom is not that which comes down from above, but is earthly, sensual, and demonic.
16: For where jealousy and selfish ambition are, there is confusion and every evil deed.
17: But the wisdom that is from above is first pure, then peaceful, gentle, reasonable, full of mercy and good fruits, without partiality, and without hypocrisy.
18: Now the fruit of righteousness is sown in peace by those who make peace.
[James 4]
1: Where do wars and fightings among you come from? Don’t they come from your pleasures that war in your members?
2: You lust, and don’t have. You murder and covet, and can’t obtain. You fight and make war. You don’t have, because you don’t ask.
3: You ask, and don’t receive, because you ask with wrong motives, so that you may spend it on your pleasures.
—–
** Main verse: 1 Peter 1:6 **
[1 Peter 1]
3: Blessed be the God and Father of our Lord Jesus Christ, who according to his great mercy caused us to be born again to a living hope through the resurrection of Jesus Christ from the dead,
4: to an incorruptible and undefiled inheritance that doesn’t fade away, reserved in Heaven for you,
5: who by the power of God are guarded through faith for a salvation ready to be revealed in the last time.
6: In this you greatly rejoice, though now for a little while, if need be, you have been grieved in various trials,
7: that the proof of your faith, which is more precious than gold that perishes, even though it is tested by fire, may be found to result in praise, glory, and honor at the revelation of Jesus Christ—
8: whom, not having known, you love. In him, though now you don’t see him, yet believing, you rejoice greatly with joy that is unspeakable and full of glory,
9: receiving the result of your faith, the salvation of your souls.
Here are the Personal Notes:
On 2026-03-19 15:50:14, I reflected: ‘King David found peace in the cave of Adullam by shifting his focus from his fear of King Saul to total reliance on God, using the isolation to pray and worship. Instead of retaliating, David found spiritual refuge through prayer, trusting God to be his defender and helper during his darkest times.’ (Related to: Psalm 57:1-7, Psalm 142:1-7, 1 Samuel 21-22)
Bible context extracted.
Personal notes extracted.
Reasoning for: ‘What does the Bible and my personal notes say about finding peace in trials?’…
Using prompt template for: gemma
End-to-end query latency: 46.28s
Sample response snippet:
It seems you’re seeking solace in the midst of trials, a yearning that resonates deeply with many. The Bible offers a powerful message about finding peace even when surrounded by chaos. Isaiah 32:1 – PASSED
======================== 9 passed in 76.34s (0:01:16) =========================
Results of Context Relevance, Faithfulness, and Answer Correctness. Originally I planned to use Ragas but Ragas wanted a specific JSON format Gemma struggled to produce. Hence, I turned to Grok to evaluate the answer. This can be implemented with an API in production.
Definitions:
Context Relevance: How relevant and useful the retrieved Bible verses + notes are to the question?
Faithfulness: Does the generated answer stick to the retrieved context without hallucinating?
Answer Correctness: How accurate, complete, and biblically sound the final response is?
1. Context Relevance: 8.5 / 10
Strengths:
The response correctly pulls and references several relevant verses that actually appear in your retrieval log: Isaiah 32:17, Romans 5:1, Matthew 10:13, James 3:18, 1 Peter 1:6. It ties the theme of peace to trials/suffering reasonably well. It incorporates the personal note about David in the cave of Adullam.
Weaknesses:
It includes Hebrews 3:8, which is weakly relevant (it mentions “trial” but is actually a warning against hardening hearts, not about finding peace).
It brings in Jeremiah 8:11 (“peace, peace” when there is no peace), which is not in the retrieved contexts and is only loosely related.
Overall, it does a decent job using the retrieved context but includes some drift.
Verdict: Good relevance, but could be tighter.
2. Faithfulness: 7.0 / 10
Strengths:
Most of the core ideas are grounded in the verses mentioned. It doesn’t wildly hallucinate new verses.
Weaknesses (Main Issues):
It attributes ideas to verses that don’t strongly support them. For example: Matthew 10:13 is about offering peace when entering a house — the response stretches it into “offering peace even when it isn’t reciprocated.” Jeremiah 8:11 is not about finding peace in trials — it’s a condemnation of false prophets.
Some interpretive flourishes go beyond what the text directly says (e.g., “This isn’t just a passive state; it’s an active result…”).
Verdict: Moderately faithful, but has noticeable interpretation creep and uses at least one non-retrieved verse.
3. Answer Correctness (Theological Accuracy + Helpfulness): 8.7 / 10
Strengths:
Overall tone is compassionate, reflective, and biblically grounded — very much in line with your desired “heart-to-heart, respectful” style. Core message is correct: Peace in trials comes from relationship with God, trust, righteousness, and future hope. Good integration of David’s story and 1 Peter 1:6 (a very strong verse for this topic). Ends on a solid note with Romans 8:28 (even though it wasn’t retrieved, this is a very appropriate verse).
Weaknesses:
Slightly wordy and repetitive in places. Minor over-interpretation of some verses.
Verdict: Strong on correctness and pastoral tone.
Overall Score: 8.1 / 10
Summary:
This is a good response for a local RAG system. It feels spiritually encouraging and mostly stays on topic. The biggest room for improvement is tighter faithfulness — reducing drift into loosely related verses and sticking more closely to the strongest retrieved passages (especially John 16:33, Isaiah 32:17, Romans 5:1, and 1 Peter 1:6). These can be greatly improved with a larger model with access to more GPU capacity. Going beyond 9 B parameters is not practical with my current system.