IsnadGuard
Hadith Chain Defect Detection System
From understanding the Islamic domain and computer science foundations, to downloading your dataset, designing your system, and submitting all four assignments — this guide covers everything.
What is a Hadith?
A Hadith (حديث — plural: Ahadith) is a recorded account of something the Prophet Muhammad ﷺ said, did, or silently approved of. After the Holy Quran, the Ahadith form the second most important foundation of Islamic guidance and law. For over fourteen centuries, Muslim scholars have treated the verification of these narrations as both a scholarly duty and a religious obligation — because a fabricated or distorted statement attributed to the Prophet ﷺ would constitute a grave religious harm.
Every Hadith you encounter in any of the six major Hadith collections — Sahih al-Bukhari, Sahih Muslim, Sunan Abu Dawud, Jami al-Tirmidhi, Sunan al-Nasa'i, and Sunan Ibn Majah — has two parts:
- Isnad (إسناد) — The chain of narrators who transmitted the Hadith from one generation to the next, tracing back to someone who heard or witnessed it directly.
- Matn (متن) — The actual content — what the Prophet ﷺ said or did.
How Scholars Grade a Hadith
Not every statement attributed to the Prophet ﷺ is authentic. Over the centuries, some narrations were fabricated by people with political or religious agendas, and others were simply misremembered or confused. Islamic scholars developed a rigorous science called Ilm al-Hadith — the science of Hadith criticism — to evaluate each narration systematically. The grade assigned to a Hadith depends primarily on the quality of its narrator chain:
| Grade | Arabic | Meaning | What it implies for the chain |
|---|---|---|---|
| Sahih | صحيح | Authentic | Unbroken chain, every narrator reliable and precise in memory, no hidden defects |
| Hasan | حسن | Good | Meets most criteria; one narrator is slightly weaker in precision than ideal |
| Da'if | ضعيف | Weak | Chain has at least one structural defect or one unreliable narrator |
| Mawdu' | موضوع | Fabricated | A known liar appears in the chain — the Hadith is completely rejected |
- In your own words, explain to a non-Muslim friend what an Isnad is and why it matters, using an analogy from software engineering — for example, a version control history or an audit log.
- Why would a "fabricated" Hadith be more harmful than simply a "weak" one? What does this tell you about why detection matters?
The Isnad — Understanding the Chain Structure
The word Isnad (إسناد) comes from the Arabic root meaning "to support" or "to lean upon." It is the ordered sequence of people who transmitted a Hadith from the Prophet ﷺ forward through time until it was recorded in a book. Each person in the chain heard the narration from the person before them and passed it to the person after them.
In computational terms, you can think of an Isnad as a directed linked list — each narrator is a node, each transmission is an edge pointing from the source toward the recorder. The list must be complete from beginning to end. A missing node anywhere in the chain is a structural defect.
How a Chain Looks
The chain below represents a simplified version of a real Isnad. Reading it left to right, it shows how the narration was passed forward in time:
Each arrow means "transmitted to." Bukhari heard from Yahya, who heard from Muhammad ibn Ibrahim, and so on back to the Companion who heard from the Prophet ﷺ directly.
Narrator Generations — Tabaqat
Every narrator belongs to a historical generation called a Tabaqah (طبقة — plural: Tabaqat). The generation number is absolutely critical for your defect detection system. Two consecutive narrators in a chain must belong to adjacent or near-adjacent generations — because they must have been alive at the same time to transmit to each other. If your data shows that Narrator A is from Generation 3 and Narrator B, who supposedly heard directly from A, is from Generation 7, that four-generation gap is your primary signal that someone is missing between them.
| Generation | Arabic Term | Description |
|---|---|---|
| 0 | — | Prophet Muhammad ﷺ — the ultimate source |
| 1 | Sahaba (صحابة) | Companions — personally met the Prophet ﷺ |
| 2 | Tabi'un (تابعون) | Successors — met the Companions |
| 3 | Tabi al-Tabi'in | Successors of Successors |
| 4–9 | Later generations | Transmission through subsequent scholarly generations |
| ~9–12 | Collectors | Major Hadith book authors — Bukhari, Muslim, Abu Dawud, etc. |
- A Companion (Generation 1) lived in the 7th century CE. A narrator from Generation 7 would have lived approximately 150–200 years later. Is it possible for Generation 7 to have heard directly from Generation 1 without anyone in between? What does this tell your system?
- Before you start coding anything, sketch a paper diagram of how you would represent a narrator chain as a data structure in Python — what type would the chain be, and what fields would each narrator object contain?
Types of Chain Defects — What Your System Must Detect
A defect in an Isnad is called an Illah (عِلَّة) — a hidden weakness. Islamic scholars spent centuries classifying these defects and developing systematic methods to identify them. Your project takes this classical taxonomy and implements it computationally. There are five main defect types your system must handle:
One narrator is missing from somewhere in the middle of the chain. Detected by finding a generation gap of exactly one between two consecutive narrators — meaning one person who should appear between them has been omitted.
A Successor (Generation 2) narrates directly from the Prophet ﷺ, skipping the Companion (Generation 1) entirely. The very first link is missing. Detected by checking whether any Generation 1 narrator appears in the chain.
Two or more consecutive narrators are missing — the generation gap between two narrators is two or more. This is a more severe form of Munqati'. Both must be detected separately, as they have different implications.
The chain stops at a Companion and does not reach back to the Prophet ﷺ — the narration is attributed to the Companion's own statement. Detected by checking whether the final narrator in the chain is a Generation 1 figure.
The chain is structurally complete but contains a narrator whose reliability grade is Da'if (weak), Majhul (unknown), or lower. The problem is not the structure but the trustworthiness of a person within it.
Example — A Broken Chain (Munqati')
The gap from Gen 3 to Gen 7 signals Mu'dal (four generations missing). Narrator C's Da'if grade signals an additional weak narrator defect. Both are present in the same chain.
- If a chain has both a Munqati' gap and a weak narrator, which defect is more serious? How would your system communicate both findings to the user?
- For your SRS functional requirements, write the requirement for Munqati' detection using the correct format: "The system shall..." — without mentioning Python or any library name.
Machine Learning — The Core CS Discipline Behind This Project
Machine Learning (ML) is a branch of computer science in which systems learn patterns from data rather than following explicitly programmed rules. Instead of a programmer writing every decision, a machine learning model is trained on examples and learns to generalize from them. In the context of your project, you are training a model to look at the features of a narrator chain and predict its grade — Sahih, Hasan, or Da'if.
The Three Types of Machine Learning
Your project uses supervised learning, so understanding how the three types differ gives you the necessary context to explain your choice in the SRS:
| Type | Description | Used in this project? |
|---|---|---|
| Supervised Learning | The model is trained on labeled data — each example has a known correct answer. The model learns the relationship between input features and labels. | Yes — chain grades (Sahih/Da'if) are the labels |
| Unsupervised Learning | No labels — the model finds hidden patterns or groupings in the data on its own (e.g., clustering). | Optionally, for clustering narrator types |
| Reinforcement Learning | An agent learns by interacting with an environment and receiving rewards or penalties. | Not used in this project |
Classification — What Your ML Module Does
Within supervised learning, your task is a classification problem — assigning each chain to one of several categories. The algorithm looks at a set of input features you provide (numbers derived from the chain structure) and maps them to an output category (the chain's grade).
The Algorithms You Will Compare
Your Module 6 requires you to train and compare at least two classification algorithms using the Scikit-learn library. Here is what each one does conceptually:
| Algorithm | How it works conceptually | Strengths for this project |
|---|---|---|
| Decision Tree | Builds a tree of if/then rules from the training data. Each branch represents a decision based on one feature value. | Very interpretable — you can read the tree and understand exactly what rules it learned. Good for a first model. |
| Random Forest | Builds hundreds of decision trees, each trained on a slightly different random subset of the data, and takes the majority vote. | Much more accurate than a single tree. Resistant to overfitting. Also provides feature importance scores. |
| Logistic Regression | Despite the name, it is a classification algorithm. It calculates the probability that a chain belongs to each grade class based on a weighted combination of its features. | Fast, efficient, good baseline. Works well when features are reasonably linear. |
Key Evaluation Metrics You Must Know
After training, you measure how well your model performs on data it has never seen before (the test set). Your SRS should mention these metrics as non-functional requirements, and your Design Document must show them in your test cases:
| Metric | What it measures | Plain-English meaning |
|---|---|---|
| Accuracy | Overall correct predictions / total predictions | Out of all chains I classified, what percentage did I get right? |
| Precision | True positives / (true positives + false positives) | Of the chains I labeled Da'if, how many were actually Da'if? |
| Recall | True positives / (true positives + false negatives) | Of all the truly Da'if chains, how many did I catch? |
| F1 Score | Harmonic mean of Precision and Recall | A balanced measure when your classes are unequal in size |
| Confusion Matrix | A table showing correct vs. incorrect predictions per class | Shows exactly which classes are being confused with which others |
- If your model has 90% accuracy but only detects 40% of the actually Da'if chains, which metric is more important for your use case — accuracy or recall? Explain why.
- Why do you need to split the dataset into training and testing sets? What would happen if you trained and tested on the same data?
- In your Design Document Test Cases section, write one test case for the ML classifier. What is the input, what is the expected output, and what metric confirms success?
Data Science — The Pipeline That Connects Everything
Data Science is the discipline of extracting meaningful insights from structured and unstructured data through a combination of statistics, programming, and domain knowledge. It is not a single technique but a pipeline — a sequence of steps that transforms raw data into useful knowledge. Your entire project follows this pipeline from beginning to end.
The Data Science Pipeline in Your Project
Every module you build corresponds to a stage in this pipeline. Understanding the pipeline as a whole helps you see why the modules are ordered the way they are, and why skipping a step causes problems downstream:
| Pipeline Stage | Your Project Module | What goes wrong if you skip it |
|---|---|---|
| Data Collection | Module 1 — loading hadith-kg | You have no data to work with |
| Data Cleaning | Module 1 — handling nulls, standardizing grades | Missing values crash your algorithms; inconsistent labels produce wrong classifications |
| Exploratory Data Analysis | Module 5 — statistics and charts | You build a model without understanding your data — you cannot explain your results in the viva |
| Feature Engineering | Module 6 — extracting numerical features from chains | The ML model has nothing meaningful to learn from — raw names are not numbers |
| Modeling | Module 6 — training classifiers | No predictive capability |
| Evaluation | Module 6 — accuracy, F1, confusion matrix | You cannot claim your model works; your test cases have no basis |
| Interpretation | Module 7 — SHAP explainability | You have a black-box prediction with no explanation — useless for scholars |
| Deployment | Module 8 — Streamlit dashboard | Your system only runs in a notebook; no real user can use it |
Feature Engineering — The Most Important Step You Will Do
Machine learning algorithms work with numbers, not names or text. The process of converting raw data into meaningful numerical inputs for a model is called feature engineering, and it is arguably the most important skill in data science. For this project, you must convert each narrator chain into a set of numerical features. Here are the features you should design:
| Feature Name | How to Compute It | Why It Matters |
|---|---|---|
chain_length | Count the number of narrators in the chain | Unusually short chains may signal missing narrators |
max_gen_gap | Largest generation difference between any two consecutive narrators | Directly signals Munqati' or Mu'dal severity |
total_gen_gaps | Count of all consecutive pairs with a generation gap > 1 | Measures how many breaks exist, not just the largest one |
weak_narrator_count | Number of narrators whose grade is Da'if, Majhul, or worse | Directly signals the weak narrator defect |
min_grade_score | Encode grades as numbers (Thiqah=3, Hasan=2, Da'if=1, Majhul=0, etc.) and take the minimum across the chain | The weakest link determines the chain's overall quality |
has_companion | Binary: 1 if a Generation 1 narrator is in the chain, 0 if not | Absence indicates Mursal defect |
ends_at_companion | Binary: 1 if the last narrator is Generation 1, 0 if not | Indicates Mawquf — chain does not reach the Prophet ﷺ |
- Can you think of one additional feature that might help the ML model — something not listed in the table above? Explain your reasoning.
- If the
min_grade_scorefeature is 0 (Majhul) but the chain has no structural gaps, what grade would you expect the model to predict? Why? - For your Data Pipeline Diagram in the Design Document, draw the flow from the raw database through all the steps above to the final prediction. How many steps are in the pipeline?
Explainable AI — Why It Matters More Here Than Anywhere
Explainable AI, commonly abbreviated as XAI, is a field of artificial intelligence focused on making the decisions of machine learning models understandable to humans. When a model classifies an image as a cat, most users are satisfied with that answer. But when a model says "this Hadith chain is Da'if," a scholar, a researcher, or anyone taking that verdict seriously will immediately ask: why? Which part of the chain caused this classification? Was it a weak narrator, a broken link, or something else?
Without an explanation, your model is a black box — it produces an answer but cannot justify it. In the domain of Hadith authentication, an unjustified verdict is essentially useless, because the entire tradition of Hadith science is built on transparency of reasoning. This is precisely why the XAI module is not optional in your project — it is what makes the system academically credible and practically meaningful.
SHAP — The Method You Will Use
SHAP stands for SHapley Additive exPlanations. It is a mathematically grounded method for explaining individual predictions made by any machine learning model. The core idea comes from game theory: each feature of your input (each number in the chain's feature vector) gets a "Shapley value" that represents how much it contributed to pushing the prediction in a particular direction.
Concretely, for a chain that your model classifies as Da'if, SHAP might produce an explanation like: "The max_gen_gap feature contributed +0.42 toward the Da'if prediction, while weak_narrator_count contributed +0.31, and chain_length contributed -0.08 (slightly away from Da'if)." This tells a scholar exactly what structural feature was most responsible for the verdict.
- If SHAP tells you that
has_companion = 0(absence of a Companion) was the most important feature for a Da'if prediction, which defect type does that correspond to? What would you tell the user on screen? - Why is SHAP particularly important in the context of Hadith authentication compared to, say, spam email detection?
Graph Theory — How Narrator Chains Become Visual Networks
Graph theory is a branch of mathematics and computer science dealing with structures made up of nodes (also called vertices) connected by edges. Many real-world problems map naturally to graphs: social networks, road maps, web links, electrical circuits, and — as your project demonstrates — narrator transmission chains.
How a Narrator Chain Becomes a Graph
In your project, every narrator is a node and every transmission relationship (A heard from B) is a directed edge. The chain from the Prophet ﷺ through multiple narrators to the collector is a directed path graph — a sequence of nodes where edges only go in one direction. Your visualization module renders this graph on screen, coloring each node by the narrator's reliability grade and marking broken links with a distinct visual style.
The Library You Will Use
NetworkX is the standard Python library for creating, analyzing, and drawing graphs. It supports directed and undirected graphs, weighted edges, and various layout algorithms for positioning nodes on screen. For rendering, it works together with Matplotlib. You do not need to implement graph theory from scratch — NetworkX handles the data structure and Matplotlib handles the visual output.
What Graph Features Are Useful for Your Project
Beyond visualization, graph analysis can contribute to your ML feature set. Concepts like node degree (how many chains a narrator appears in), betweenness centrality (how many chains pass through a narrator), and connected components (whether the chain forms one continuous path) are all computable from graph structures and may be worth exploring as additional features in Module 6.
- In your class diagram for the Design Document, which class is responsible for building the graph object from the chain data? What methods would it need?
- A narrator who appears in 5,000 different chains has a high node degree. Does that make them more or less likely to be reliable? Think about this before deciding whether to include it as a feature.
Which Datasets to Use and Why
All datasets recommended for this project are peer-reviewed, publicly available, and free. No unofficial sources, no Kaggle datasets from anonymous contributors, and no data that requires any payment. The following three are your primary options:
| Dataset | Size | Format | Best For |
|---|---|---|---|
| hadith-kg (emadjumaah) | 577K chains, 49K narrator biographies | SQLite database (.db) | Primary dataset — narrator grades already labeled, generation numbers included, chains fully linked. Use this for everything. |
| Sanadset 650K (arbml) | 650,986 Hadith records | Parquet / CSV | Additional chain records for testing. Use this if you want to test your detection system on a larger volume of chains. |
| MIS Dataset (Sahih Muslim) | 7,748 Hadith, 77,797 narrator links | Excel (.xlsx) | Focused, clean dataset for Sahih Muslim specifically. Useful for prototyping and initial testing. |
Jumaah, E. (2024). hadith-kg: A Knowledge Graph of the Sciences of Hadith. Hugging Face Datasets. DOI pending — cite as: https://huggingface.co/datasets/emadjumaah/hadith-kg
Al-Thubaiti, K. et al. (2022). Sanadset 650K: Data on Hadith Narrators. Data in Brief, 44, 108540. DOI: 10.1016/j.dib.2022.108540
Key Tables in hadith-kg
| Table | What It Contains | Key Columns |
|---|---|---|
rawis | Narrator biographies — the Rijal data | id, name, grade, tabaqah, death_year, is_stub |
sanads | Complete isnad chains with overall grade | id, hadith_id, grade |
sanad_rawis | Junction — maps narrators to chains with position order | sanad_id, rawi_id, pos |
hadiths | Hadith text (matn) and collection reference | id, collection_id, hadith_no, matn |
collections | The Hadith books (Bukhari, Muslim, etc.) | id, name, author |
Using Hugging Face — What It Is and How to Get Started
Hugging Face is the world's largest open-source platform for sharing AI models and datasets. Think of it as GitHub, but purpose-built for machine learning. The platform hosts over 500,000 public datasets, all freely accessible. Your primary dataset — hadith-kg — is hosted there.
Creating Your Account
Register at huggingface.co
Go to https://huggingface.co and click Sign Up. You can register with any email address — Gmail, Yahoo, or your VU email. No payment is required. A free account gives you full access to all public datasets.
Browse the Dataset
After logging in, go directly to https://huggingface.co/datasets/emadjumaah/hadith-kg — or search "hadith-kg" in the Datasets section. The Dataset Card shows a description and the available files. Click the "Files and versions" tab to see the actual files.
Generate an Access Token
For downloading through Python, you need a read-access token. Go to your account Settings → Access Tokens → New Token. Name it "isnadguard" and select "Read" permission. Copy the token and keep it safe — you will use it once during the download setup.
Downloading: The Windows Method
If you prefer a graphical, point-and-click approach to downloading the dataset file, this is the method for you. It requires no programming, just a browser and the Hugging Face website.
Log in and Navigate to the Dataset
Go to https://huggingface.co/datasets/emadjumaah/hadith-kg in your browser while logged into your Hugging Face account.
Click "Files and versions"
On the dataset page, click the "Files and versions" tab. You will see a list of files in the repository. The file you need is hadith-kg.db — approximately 1.6 GB in size.
Click the Download Icon
Next to hadith-kg.db, click the download icon (↓). Your browser will begin downloading the file to your Downloads folder. Do not close the browser tab until the download completes.
Move the File to Your Project Folder
After downloading, move hadith-kg.db to a folder named data inside your project directory. Keep a backup copy somewhere safe — perhaps an external drive or Google Drive — so you never have to download it again.
Downloading: The Python Code Method
Downloading through Python gives you more control, better progress visibility, and — most importantly — the ability to resume an interrupted download. This is the recommended method for most students.
Step 1 — Install the Required Library
Open your terminal or command prompt and run this command once. You do not need to run it again after it succeeds:
pip install huggingface_hub
Step 2 — Log In to Hugging Face
Run this command in your terminal. It will ask you to paste the access token you generated in Chapter 9. You only need to do this once — your credentials are saved to your machine after the first login:
huggingface-cli login
Step 3 — Download the Dataset File
Create a new Python file in your project folder — for example, download_dataset.py — and write the following code exactly as shown. Run it with python download_dataset.py in your terminal:
from huggingface_hub import hf_hub_download import os # Create the data folder if it does not exist os.makedirs("./data", exist_ok=True) print("Starting download of hadith-kg.db ...") print("File size: approximately 1.6 GB. This may take a while.") print("If interrupted, re-run this script — the download will resume.") file_path = hf_hub_download( repo_id="emadjumaah/hadith-kg", filename="hadith-kg.db", repo_type="dataset", local_dir="./data", local_dir_use_symlinks=False ) print(f"Download complete. File saved at: {file_path}") print(f"File size on disk: {os.path.getsize(file_path) / (1024**3):.2f} GB")
local_dir parameter is what makes resuming possible. When you set a local directory, the Hugging Face library saves a partial file with a .incomplete extension. If your connection drops and you re-run the script, the library detects this partial file and resumes from where it stopped — you do not lose any progress. This is the key advantage over a browser download.Downloading the Sanadset 650K (Alternative / Additional)
To download the Sanadset dataset instead or in addition, replace the repo_id and filename in the script above with the following values. Sanadset is stored in Parquet format, which is more efficient than CSV for large datasets:
from huggingface_hub import snapshot_download # This downloads all Parquet files of the Sanadset into ./data/sanadset/ snapshot_download( repo_id="arbml/Sanadset", repo_type="dataset", local_dir="./data/sanadset", local_dir_use_symlinks=False )
Resumable Downloads — What to Do When the Connection Drops
Internet connections in Pakistan are often unstable, and a 1.6 GB file can take a long time to download. Here is exactly what happens in each scenario and what you should do:
| Scenario | What happens | What to do |
|---|---|---|
| Browser download interrupted | Partial download is usually discarded. The file in your Downloads folder is incomplete and unusable. | Delete the partial file and restart the download from the beginning. Consider switching to the Python method. |
| Python script interrupted | A .incomplete partial file is saved in ./data/. No progress is lost. | Re-run the same Python script. The library will detect the partial file automatically and resume from where it stopped. |
| Python script completes but file seems wrong | The download may have been corrupted partway through without the library detecting it. | Check the file size in the print output — it should be approximately 1.60–1.65 GB. If it is much smaller, delete the .db file (not the .incomplete file) and re-run the script. |
Using the Streaming Option — No Download Required
If downloading is genuinely impossible on your connection, you can stream just the records you need directly from Hugging Face into your Python session — no file is saved to disk at all. This is suitable for the Sanadset dataset but not for hadith-kg (which is a SQLite file and must be opened locally). Here is how streaming works for Sanadset:
from datasets import load_dataset # Install the datasets library first: pip install datasets # streaming=True means no file is downloaded — data streams record by record ds = load_dataset( "arbml/Sanadset", streaming=True, split="train" ) # Take only the first 1000 records for your prototype work sample = list(ds.take(1000)) import pandas as pd df = pd.DataFrame(sample) print(df.head()) print("Columns:", df.columns.tolist()) print(f"Loaded {len(df)} records")
Exploring the Dataset — What to Do Before Writing Any Module
The first thing you must do after successfully connecting to hadith-kg.db is explore the data. Do not jump to building the defect detector until you have spent time understanding what is actually in the database. Here is the exploration code to run:
import sqlite3 import pandas as pd # Open connection to the database conn = sqlite3.connect("./data/hadith-kg.db") # ── Step 1: List all tables ────────────────────────────── tables = pd.read_sql_query( "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name", conn ) print("=== Tables in database ===") print(tables) # ── Step 2: See all narrator grade types ───────────────── grades = pd.read_sql_query( "SELECT grade, COUNT(*) as count FROM rawis GROUP BY grade ORDER BY count DESC", conn ) print("\n=== Narrator Grade Distribution ===") print(grades) # ── Step 3: See generation distribution ────────────────── tabaqat = pd.read_sql_query( "SELECT tabaqah, COUNT(*) as count FROM rawis WHERE tabaqah IS NOT NULL GROUP BY tabaqah ORDER BY tabaqah", conn ) print("\n=== Generation (Tabaqah) Distribution ===") print(tabaqat) # ── Step 4: Sample a complete chain ────────────────────── chain_query = """ SELECT sr.pos, r.name, r.grade, r.tabaqah, r.death_year FROM sanad_rawis sr JOIN rawis r ON sr.rawi_id = r.id WHERE sr.sanad_id = 1 ORDER BY sr.pos ASC """ chain = pd.read_sql_query(chain_query, conn) print("\n=== Sample chain — sanad_id = 1 ===") print(chain) # ── Step 5: Check chain length distribution ─────────────── lengths = pd.read_sql_query( "SELECT sanad_id, COUNT(*) as length FROM sanad_rawis GROUP BY sanad_id ORDER BY length DESC LIMIT 20", conn ) print("\n=== Top 20 longest chains ===") print(lengths) conn.close() print("\nExploration complete.")
System Inputs and Outputs — What the User Gives and Gets
Before designing any class or writing any logic, you must know precisely what the user interacts with. This section describes the inputs and outputs for every major module of your project. This directly informs your Use Case Diagram in the SRS and your Sequence Diagrams in the Design Document.
Module 1 — Data Loading
User Input
- Path to the local hadith-kg.db file, entered through a file-selection button
- Or: choice to use streaming (if no local file)
System Output
- Summary panel: total narrators loaded, total chains loaded, grade distribution
- List of any records with missing generation numbers or grades
- Confirmation that data is ready for all other modules
Module 3 — Defect Detection
User Input
- A chain ID (Sanad ID) — a number — entered in a search box
- Or: selection from a browsable list of all chains
System Output
- Ordered list of narrators with name, grade, and generation for each
- Defect report: which of the five defect types were found
- For each defect: position in chain and plain-English explanation
- Chain visualization as a graph diagram
- Overall verdict: Structurally Sound / Defective
Module 6 — ML Classifier
User Input
- A chain ID to classify
- Selection of model (Decision Tree / Random Forest / Logistic Regression)
System Output
- Predicted grade (Sahih / Hasan / Da'if)
- Confidence percentage for the prediction
- Comparison with the classical scholarly grade stored in the database
Module 7 — XAI Explanation
User Input
- Same chain ID that was classified in Module 6
System Output
- Feature importance bar chart for this specific chain
- Which feature contributed most to the prediction
- Written plain-English explanation linking the dominant feature back to the defect it represents
Module 8 — Final Dashboard
User Input
- A chain ID entered in a search box, or selected from a list
System Output
- All outputs from Modules 3, 4, 6, and 7 in a single integrated view
- One-click button to export everything as a PDF report
Implementation Logic — Pseudocode for Core Functions
This chapter gives you the logical skeleton of each core function as pseudocode. Pseudocode describes what the function should do in plain English, without being tied to any specific programming language. Your task is to take this pseudocode and implement it yourself in Python, making decisions about data types, error handling, and edge cases along the way.
Building a Chain from the Database
FUNCTION build_chain(chain_id): -- Query the database -- -- JOIN sanad_rawis with rawis WHERE sanad_id = chain_id -- -- SELECT pos, name, grade, tabaqah, death_year -- -- ORDER BY pos ASC -- IF no rows returned: raise an error: "Chain ID not found" FOR EACH row in results: create a narrator object with: name, grade, tabaqah, death_year, position RETURN the ordered list of narrator objects -- Edge cases to handle in your implementation: -- -- What if tabaqah is NULL for some narrators? -- -- What if grade is NULL? -- -- What if chain_id is not an integer? --
Munqati' Detection — Broken Chain
FUNCTION detect_munqati(narrator_list): gaps_found = empty list FOR each pair (narrator_A, narrator_B) where B immediately follows A: IF either narrator has no tabaqah value: skip this pair, record as "generation data unavailable" CONTINUE gap = absolute_value(narrator_B.tabaqah - narrator_A.tabaqah) IF gap is exactly 1: record this as a Munqati' gap note: which two narrators, gap size = 1 ELSE IF gap is greater than 1: -- Do NOT count this here — it is Mu'dal, handled separately -- CONTINUE RETURN detected = (gaps_found is not empty), details = gaps_found
Mu'dal Detection — Multiple Narrators Missing
FUNCTION detect_mudal(narrator_list): severe_gaps = empty list FOR each pair (narrator_A, narrator_B) where B immediately follows A: IF either has no tabaqah: skip gap = absolute_value(narrator_B.tabaqah - narrator_A.tabaqah) IF gap >= 2: record as Mu'dal gap note: which narrators, how many appear to be missing (gap - 1) RETURN detected = (severe_gaps is not empty), details = severe_gaps
Mursal Detection — Missing Companion
FUNCTION detect_mursal(narrator_list): has_companion = False has_successor = False FOR EACH narrator in narrator_list: IF narrator.tabaqah == 1: has_companion = True IF narrator.tabaqah == 2: has_successor = True is_mursal = (has_companion == False) AND (has_successor == True) RETURN detected = is_mursal, explanation = "Tabi'i narrates without a Companion in chain" if is_mursal else "Companion present in chain"
The Master Analysis Function
FUNCTION analyze_chain(chain_id): narrator_list = build_chain(chain_id) IF narrator_list is empty or error: RETURN error to user: "Chain not found" results = empty dictionary results["munqati"] = detect_munqati(narrator_list) results["mudal"] = detect_mudal(narrator_list) results["mursal"] = detect_mursal(narrator_list) results["mawquf"] = detect_mawquf(narrator_list) results["weak_narrator"] = detect_weak_narrator(narrator_list) any_defect_found = check if ANY result has detected = True overall_verdict = "Defective Chain" if any_defect_found else "No structural defects detected" RETURN narrator_list, results, overall_verdict -- Remember: detect_mawquf and detect_weak_narrator pseudocode -- -- must be written by YOU, using the concepts from Chapter 3. --
SRS Assignment — Software Requirements Specification
The SRS is your first formal assignment and the foundation of your entire project. It describes WHAT your system will do — from the perspective of the user — without describing HOW it will do it. You should not mention Python, Scikit-learn, Pandas, SQLite, or any technology in the functional requirements section. The SRS template you have been given contains six sections. Each is explained in detail below.
Section 1 — Scope of the Project
Write a comprehensive description of what IsnadGuard does, why it exists, what problem it solves, who will use it, and what it does not do. The scope should be specific enough that a reader can tell exactly what is and is not part of your system. Include the domain background — explain briefly what an Isnad is and why defect detection matters — so that any evaluator understands the context even without prior Islamic studies knowledge. Refer to Chapters 1 through 3 of this guide for the domain content.
Section 2 — Functional and Non-Functional Requirements
This is the most technically important section of your SRS. Functional requirements describe specific behaviors your system must perform. Every requirement must start with "The system shall..." — this is the IEEE 830 standard phrasing. Non-functional requirements describe quality attributes — how well the system performs, how secure it is, how usable it must be.
Examples of correctly written functional requirements for IsnadGuard:
- The system shall allow the user to load the Hadith narrator chain database from a local file path.
- The system shall reconstruct a complete narrator chain for any given chain identifier from the loaded database.
- The system shall detect and report the presence of a Munqati' defect by identifying generation gaps between consecutive narrators.
- The system shall display each narrator's name, reliability grade, and generation number in the chain view.
- The system shall train a classification model on extracted chain features and display accuracy, precision, recall, and F1 score on the test set.
- The system shall provide an explanation for each chain grade prediction showing the contribution of each input feature.
Examples of non-functional requirements:
- The system shall display defect detection results within three seconds of receiving a chain identifier.
- The system shall handle chains of up to fifty narrators without performance degradation.
- The system's interface shall be usable by a user with no prior knowledge of Hadith science — all technical terms shall be accompanied by plain-English explanations.
Section 3 — Use Case Diagram
A Use Case Diagram shows the actors in the system and the actions they can perform. It is a high-level overview — it does not show steps or sequences, only capabilities.
Draw one overall diagram with two actors: User (the student or researcher running the application) and System (the application itself, which is not explicitly drawn but implied). Use cases include: Load Dataset, View Statistics, Search Chain, View Defect Report, View Chain Graph, Classify Chain, View Explanation, Export Report. Connect each actor to the use cases they can initiate.
draw.io — UML → Use Case shapes CS504 reference: Lecture 6 — Use Case Diagrams with worked examplesSection 4 — Usage Scenarios
A Usage Scenario is a structured description of one complete use case. The template requires you to write a scenario for every use case shown in the diagram. Each scenario is presented as a table with these fields:
| Field | Description | Example for "Search Chain" |
|---|---|---|
| Use Case Title | The name of the use case | Search Chain by Identifier |
| Use Case ID | A unique reference code | UC-03 |
| Actor | Who initiates this use case | User |
| Description | What the use case accomplishes | The user enters a chain ID and the system retrieves and displays the full narrator chain with all biographical attributes. |
| Pre-Conditions | What must be true before this use case begins | The dataset has been successfully loaded (UC-01 is complete). |
| Actions (Main Path) | The normal, successful sequence of steps | 1. User enters a chain ID in the search box. 2. User clicks Analyze. 3. System retrieves chain from database. 4. System displays narrator list and runs defect detection. |
| Alternative Paths | Valid variations of the normal path | User selects chain from a browsable list instead of typing the ID. |
| Exceptions | Error conditions | Chain ID does not exist — system displays "No chain found with this identifier." |
| Post-Conditions | What is true after the use case completes | Defect report and chain visualization are displayed on screen. |
| Author | Your student ID | Your VU student ID |
Section 5 — Adopted Methodology
Your project follows the VU Process Model, which combines elements of the Waterfall and Spiral models. Explain this choice specifically for your project: the Waterfall component means you follow sequential phases (requirements → design → implementation → testing → delivery), while the Spiral component means you revisit and refine each phase incrementally — for example, your prototype in Assignment 3 is an early spiral iteration, and your Final Project is the complete spiral. Reference CS605 (SE2) for the formal methodology description. Also mention that the four-assignment structure (SRS, Design Document, Prototype, Final) maps directly to the phases of this combined model.
Section 6 — Work Plan (Gantt Chart)
Create a Gantt chart using Microsoft Project or any free alternative (GanttProject, TeamGantt free tier, or even Excel's bar chart feature). The chart must show all eight modules of your project, organized across two semesters, with the four assignment submission dates marked as milestones. Each module should have a start date, an end date, and its corresponding assignment deliverable. Export the chart as PNG and insert it into your SRS document.
Design Document — How the System Is Built
The Design Document is your second assignment and describes HOW your system works internally. Where the SRS said "the system shall classify chains," the Design Document explains what classes implement this, what database queries are used, how the UI is structured, and what the data flow looks like. The template you have been given contains eight sections.
Section 1 — Introduction to the Design Document
Describe what this document contains and why the Design phase matters in a software development project. Explain that design decisions made here — class structure, database access strategy, module dependencies — prevent costly mistakes during implementation. State which design standards you followed (IEEE 1016 for design documents, which CS504 Lectures 23–25 cover). Briefly list the eight sections of the document and summarize what each one contributes.
Section 2 — Entity Relationship Diagram (ERD)
The ERD shows the structure of your data — the tables in hadith-kg.db, their attributes, and the relationships between them. You must show all five tables: rawis, sanads, sanad_rawis, hadiths, and collections. For each table, show its primary key, all important attributes with their data types, and the foreign key relationships connecting the tables.
Show all five tables of hadith-kg.db with their primary keys (id fields), relevant columns, and foreign key relationships. Key relationships: rawis ↔ sanad_rawis via rawi_id; sanads ↔ sanad_rawis via sanad_id; sanads ↔ hadiths via hadith_id; hadiths ↔ collections via collection_id.
dbdiagram.io — free, clean ERDs from text CS403 reference: ERD lectures — entity types, relationships, cardinality notationSection 3 — Sequence Diagrams
A Sequence Diagram shows the order of messages and method calls between objects during one specific use case. The template requires one sequence diagram per use case from your SRS. For your project, you should draw sequence diagrams for at minimum: loading the dataset, analyzing a chain (defect detection), classifying a chain (ML), and exporting a report.
Objects involved: User, Dashboard (UI), DataLoader, ChainBuilder, DefectDetector, ChainVisualizer. Sequence: User enters chain ID → Dashboard calls ChainBuilder.build_chain(id) → ChainBuilder queries database → returns narrator list → Dashboard calls DefectDetector.analyze(narrator_list) → returns defect report → Dashboard calls ChainVisualizer.draw(narrator_list, defects) → returns graph image → Dashboard displays all results to User.
draw.io — UML → Sequence shapes CS504 reference: Lectures 23–25 — Sequence Diagrams in architectural designSection 4 — Architecture Design Diagram
The Architecture Design shows how your system is organized into layers or tiers. For IsnadGuard, use a three-tier architecture: a Presentation Layer (the Streamlit dashboard the user sees), a Business Logic Layer (your Python modules — data loader, chain builder, defect detector, ML classifier, XAI explainer, report generator), and a Data Layer (the SQLite database). Show how these layers connect and which direction communication flows.
Layer 1 (Presentation): Streamlit web interface — screens for data loading, chain search, statistics, ML prediction, and report export. Layer 2 (Business Logic): DataLoader, ChainBuilder, DefectDetector, ChainVisualizer, StatisticsEngine, MLClassifier, XAIExplainer, ReportGenerator. Layer 3 (Data): hadith-kg.db SQLite database accessed via Python's sqlite3 library.
draw.io — General → box and arrow layout CS504 reference: Lecture 23 — Architectural Views; Lecture 24–25 — Architectural ModelsSection 5 — Class Diagram
The Class Diagram shows the object-oriented structure of your Python code — the classes, their attributes, their methods, and the relationships between them. This is one of the most important diagrams in the Design Document, and your supervisor will use it to verify that you have properly planned your code architecture before implementing it.
Classes to include: DataLoader (methods: connect(), load_narrators(), load_chains(), close()); NarratorChain (attributes: sanad_id, narrator_list, length; methods: get_names(), get_grades(), get_generations()); DefectDetector (methods: detect_all(), detect_munqati(), detect_mudal(), detect_mursal(), detect_mawquf(), detect_weak()); ChainVisualizer (methods: draw_chain(), export_image()); MLClassifier (methods: extract_features(), train(), predict(), evaluate()); XAIExplainer (methods: explain(chain, model)); ReportGenerator (methods: generate_pdf()). Show relationships: Dashboard uses all these classes; MLClassifier depends on DataLoader.
draw.io — UML → Class shapes CS304 reference: OOP class diagram lectures — attributes, methods, visibility, relationshipsSection 6 — Database Design
This section goes deeper than the ERD — it provides the actual schema of the database you are using, including data types for every column, constraints (NOT NULL, PRIMARY KEY, FOREIGN KEY), and indexes. Since you are using an existing database (hadith-kg.db) rather than designing one from scratch, this section should document the relevant tables' exact schemas as they appear in the database, along with an explanation of how your application reads from each one. You can obtain the exact schema by running PRAGMA table_info(table_name) on each table in your exploration script.
Section 7 — Interface Design
Provide three to four screenshots or hand-drawn mockups of the most important screens in your application. For IsnadGuard, show: the data loading screen (file selector and summary panel), the chain analysis screen (search box, narrator list, defect report, chain graph), the ML and XAI screen (predicted grade, confidence, SHAP chart), and the export/report screen. These do not need to be final — you can update them in your Final Project submission if the actual implementation looks different.
Section 8 — Test Cases
Write test cases for each use case scenario from the SRS. Each test case must have: a test case ID, the use case it tests, pre-conditions, test input, expected output, and pass/fail criteria. For example, for the defect detection function: Input = chain with a known Munqati' gap (you can construct this from your exploration data); Expected Output = detect_munqati returns detected=True with the correct position; Pass Criteria = output matches expected and no exception is raised.
Prototype Assignment — What to Build and How to Submit
The Prototype assignment covers 30–40% of your total project work and corresponds to Modules 1 through 4. At the end of Semester 1, you must submit a working Python application that demonstrates the core data loading, chain construction, defect detection, and visualization pipeline using real hadith-kg data.
What Must Be Working in the Prototype
- Module 1: Application successfully connects to hadith-kg.db, loads narrator and chain data, displays a summary panel with total counts and grade distribution, and reports any records with missing data.
- Module 2: Given any valid chain ID, the application reconstructs the ordered narrator list with all required attributes.
- Module 3: The DefectDetector correctly identifies Munqati', Mu'dal, Mursal, Mawquf, and weak narrator defects. Test it on at least 10 different chains — 5 with known defects and 5 that are structurally sound.
- Module 4: The chain visualization renders a colored graph showing narrator grades and marking any broken links. Export works and produces a readable PNG image.
What You Submit
- A ZIP file (maximum 25 MB) containing all Python source files
- A README.txt file with step-by-step instructions to run your application — including which Python version, which libraries to install (with version numbers), and the exact command to start the application
- Do not include hadith-kg.db in the ZIP — it is too large. Your README must include instructions for downloading it
- Screenshots of each module working correctly — at least one screenshot per module
- Upload to VULMS under the Prototype Assignment submission
Final Deliverable — The Complete System
The Final Project submission is the complete, integrated, tested IsnadGuard system covering all eight modules. It must run as a single Streamlit application that a user can open, load the database, and use to analyze any chain in the dataset. Every module must be functional, tested, and connected to the others through the shared data layer.
Minimum Requirements for Full Marks
- All eight modules are complete and integrated into the Streamlit dashboard
- The ML classifier achieves at least 70% accuracy on the test set (20% holdout from hadith-kg)
- SHAP explanations are displayed for every chain classification
- The statistics dashboard (Module 5) shows at least four meaningful charts about the dataset
- PDF report generation works and produces a clean, readable document
- All code is in Python, organized into classes with proper docstrings and exception handling
- The application runs from a README.txt with a single command
What to Submit
- ZIP file with all Python source files, organized into a package structure
- A requirements.txt file listing every library with version numbers
- README.txt with complete installation and run instructions
- A sample PDF report generated by your application from a real chain
- Screenshots of every screen in the application
All Eight Modules — The Complete Picture
Connects to hadith-kg.db, loads and cleans narrator and chain data, standardizes grade labels, handles missing values, displays a summary panel. Output: clean, queryable data representation and statistics.
Semester 1 — PrototypeReconstructs individual narrator chains from the database as ordered structures with all required biographical attributes. Output: NarratorChain objects ready for detection and visualization.
Semester 1 — PrototypeImplements all five defect detection functions. Produces a structured defect report for any chain. Output: detailed report identifying defect type, position, and plain-English explanation.
Semester 1 — PrototypeRenders narrator chains as directed graph diagrams using NetworkX, colored by grade, with broken links highlighted. Output: exportable graph image for reports and on-screen display.
Semester 1 — PrototypePresents aggregate statistics — defect type frequencies, grade distributions, most-defective collections, chain length patterns — as charts and summary tables.
Semester 2 — FinalExtracts features from chains, trains and compares three classifiers (Decision Tree, Random Forest, Logistic Regression), evaluates with accuracy, F1, and confusion matrix. Output: trained model and evaluation report.
Semester 2 — FinalApplies SHAP to the trained classifier to explain each individual prediction, showing which features contributed most and linking them back to specific defect types.
Semester 2 — FinalIntegrates all modules into a Streamlit web application. User enters any chain ID and gets full defect report, chain graph, ML grade, and XAI explanation in one view. One-click PDF export.
Semester 2 — FinalWhat You Will Learn From This Project
Most final year projects teach students one or two technical skills. This project covers the full spectrum of modern data science and AI engineering — from database querying through machine learning to explainability to deployment. Every module you complete adds a skill that is genuinely in demand, both in Pakistan's growing technology sector and in graduate-level academic work internationally.
The Bigger Picture
The dataset you are working with has been referenced in multiple peer-reviewed publications on Hadith narrator analysis. The specific combination your project implements — rule-based detection plus supervised ML plus SHAP explainability — has not been published as a complete system at the time this guide was written. If you complete all eight modules seriously and write your work up carefully, this project has genuine potential as a conference paper. Your supervisor can advise on publication pathways once the project is complete.
Self-Test Questions — Verify Your Understanding
The ten questions below are the type your supervisor is most likely to ask during the viva. Try each one in your own words before revealing the answer. If you can answer all ten fluently without referring to any notes, you are genuinely prepared for the viva. If there are questions you cannot answer, return to the relevant chapter before your viva date.