Introducing Assignment Analysis: AI-Powered Feedback for Every Student, Delivered via WhatsApp

Nithun KV
Nov 7
10 min read

Innovation can make personalized learning feedback simple, instant, and accessible using WhatsApp.

The Challenge: Scaling Personalized Feedback

At TAP, over 100,000 students learn through creative, project-based assignments — each reflecting unique ideas and problem-solving approaches. But manually reviewing these artifacts at scale is unviable.

Every submission must be checked for accuracy, creativity, and effort, yet teachers spend hours decoding handwriting and applying rubrics, leading to:

Low coverage and inconsistent quality across schools
Delayed feedback that breaks the learning loop
Unstructured data that hides valuable insights

For a system serving a million learners across low-bandwidth environments, manual evaluation cannot sustain personalized growth. TAP needed a scalable AI engine to analyze multimodal assignments (text, drawings, photos), align them with learning rubrics, and deliver instant, constructive feedback via WhatsApp.

Summary

The Assignment Analysis System is a foundational component of TAP’s Personalized Adaptive Learning (PAL) ecosystem. It is an AI-driven, image-based assignment evaluation platform that analyzes student submissions and provides automated, contextual feedback directly through WhatsApp.

The system bridges the gap between student engagement and personalized learning by seamlessly integrating multiple services—Glific (WhatsApp), TAP LMS (Frappe), RabbitMQ, and a RAG-based AI analysis service—to deliver a fully automated, end-to-end feedback cycle.

Through this integration, the system:

Receives student submissions directly via WhatsApp.
Processes assignment images asynchronously through RabbitMQ for scalability and reliability.
Leverages the RAG (Retrieval-Augmented Generation) service for LLM-based reasoning and contextual interpretation of assignment content.
Returns structured insights and learning feedback to the TAP LMS.
Automatically delivers personalized feedback messages to students on WhatsApp through Glific flows.

Role in the PAL Ecosystem

Within the Personalized Adaptive Learning (PAL) architecture, the Assignment Analysis System serves as the observation and feedback engine that fuels adaptive learning loops. By continuously interpreting student work and generating individualized feedback, it provides the real-time learning signals required to:

Identify each student’s learning patterns and areas of interest.
Feed analytical insights into PAL’s Learning Profiles for deeper personalization.
Enable adaptive content recommendations and teacher guidance in future versions.
Establish a data-driven foundation for longitudinal tracking of creativity, communication, and other 21st-century skills.

Key Technologies:

Frappe Framework, RabbitMQ, LangChain, OpenAI/Together AI (Vision-enabled LLMs), Glific (WhatsApp Business API)

1. System Overview

1.1 Purpose

Enable students to submit assignment images via WhatsApp and receive automated, AI-powered analysis and feedback using vision-enabled large language models with assignment-specific context.

1.2 Key Capabilities

WhatsApp-based Submission: Students submit assignments as images through Glific flows
Vision-AI Analysis: Automated evaluation using GPT-4 Vision or Llama 3.2 90B Vision models
Asynchronous Processing: Three-stage queue architecture for scalable, non-blocking operations
Assignment-type Based Evaluation: Configurable prompt templates (Written, Practical, Performance, Collaborative)
Context-Aware Feedback: Leverages assignment metadata, learning objectives, and reference materials
Real-time Notification: Immediate feedback delivery via WhatsApp

1.3 Architecture Style

Event-Driven Architecture: AMQP message-based async communication

Microservices: Two independent Frappe applications (TAP LMS + RAG Service)
API-First Design: REST APIs for synchronous, AMQP queues for async operations
Queue-Based Workflow: submission_queue → plagiarism_results_queue → feedback_results_queue

Note: Plagiarism Check feature is currently under development

2. System Architecture

2.1 High-Level Architecture Diagram

2.2 Data Flow: Complete Submission-to-Feedback

Student Submission
- Student sends assignment image via WhatsApp
- Glific captures image and triggers webhook to TAP LMS
TAP LMS Processing
- TAP LMS receives webhook with image URL and student details
- Creates Submission document
- Publishes message to RabbitMQ [submission_queue]
RAG Service Consumption
- RAG Service consumes message from [submission_queue]
- Fetches assignment context from TAP LMS API
- Retrieves prompt template based on assignment type
AI Analysis
- Downloads image from provided URL
- Constructs prompts using system & user templates
- Calls vision-enabled LLM (GPT-4V or Llama 3.2 90B)
- Parses and validates JSON response
Feedback Publishing
- Packages feedback into standardized format
- Publishes to [feedback_results_queue] in RabbitMQ
TAP LMS Feedback Consumer
- Consumes feedback from [feedback_results_queue]
- Updates Submission document
- Triggers Glific flow to send feedback to student
Student Notification
- Glific sends formatted feedback via WhatsApp
- Student receives feedback in conversational format

3. Component Details

3.1 Glific (WhatsApp Interface)

Technology: Glific Platform https://github.com/glific

Responsibilities:

Student interaction via WhatsApp flows
Capture and forward assignment image submissions
Deliver feedback notifications to students
Webhook integration with TAP LMS

Key Features:

Flow-based conversation design
Media message handling (images)
Flow variables for dynamic content
Session management

Integration Points:

Inbound Webhook: [POST] to TAP LMS with submission data
Outbound Flow Trigger: TAP LMS calls Glific API to start feedback flow

3.2 TAP LMS (Frappe Application)

Technology: Frappe Framework (Python-based)

Git Repo: https://github.com/DalgoT4D/frappe_tap

Role: Central orchestrator and source of truth

Responsibilities:

Student and assignment data management
Curriculum and pedagogy tracking
Submission workflow orchestration
Assignment context API provider
Message routing between Glific and RAG Service
Feedback consumption and student notification

Key DocTypes:

Submission (Primary submission tracking)
- [assign_id]: Assignment identifier
- [student_id]: Student identifier (Glific contact ID)
- [img_url]: URL of submitted image
- [status]: Pending → Processing → Completed/Failed
- [overall_feedback]: Final feedback text
- [grade]: Numerical grade (if applicable)
- [created_at], [completed_at]: Timestamps
Assignment
- [assignment_name], [description], [assignment_type]
- [subject]: Link to Course Verticals
- [learning_objectives]: Table of objectives
- [reference_image]: Reference material for students
- [max_score]: Maximum possible score
- [enable_auto_feedback]: Toggle for AI feedback
- [feedback_prompt]: Custom prompt override
- [difficulty_tier]: Remedial/Basic/Intermediate/Advanced
RabbitMQ Settings (Single)
- [host], [port], [virtual_host]
- [username], [password]
- [submission_queue]: Queue name for new submissions
- [plagiarism_results_queue]: Queue consumed by RAG Service
- [feedback_results_queue]: Queue for completed feedback
Glific Settings (Single)
- [api_url], [phone_number], [password]
- [access_token], [renewal_token], [token_expiry_time]
- [webhook_secret]
Glific Flow
- [label]: “feedback”
- [flow_id]: Glific flow identifier for sending feedback

API Endpoints:

POST /api/method/tap_lms.imgana.submission.submit_artwork
- Receives submission from Glific webhook
- Parameters: [api_key], [assign_id], [student_id], [img_url]
- Creates Submission and enqueues to RabbitMQ
GET /api/method/tap_lms.imgana.submission.get_assignment_context
- Provides assignment context for RAG Service
- Parameters: [assignment_id], [student_id] (optional)
- Returns assignment details, learning objectives, student context
GET /api/method/tap_lms.imgana.submission.img_feedback
- Check submission status (used by external integrations)
- Parameters: [api_key], [submission_id]
- Returns status and feedback if completed

Business Logic:

Validates API keys for webhook authentication
Creates ImgSubmission documents
Publishes to RabbitMQ submission queue
Consumes feedback from feedback_results_queue
Updates ImgSubmission status
Triggers Glific notification flows

RabbitMQ Integration:

Publisher: Sends submission messages to [plagiarism_results_queue]
Consumer (FeedbackConsumer class):
Listens to [feedback_results_queue]
Updates ImgSubmission documents
Triggers Glific flows for student notification
Handles dead letter queue for failed messages

3.3 RabbitMQ (Message Broker)

Technology: RabbitMQ (AMQP 0-9-1)

Role: Asynchronous message transport and decoupling layer

Responsibilities:

Decouple TAP LMS from RAG Service
Provide durable work queues for async processing
Message persistence and reliability
Dead letter queue (DLQ) support for failed messages

Queue Architecture:

submission_queue (currently maps to plagiarism_results_queue)
      │
      ├──► plagiarism_results_queue ──► RAG Service Consumer
      │         (durable, persistent)
      │
      └──► plagiarism_results_queue_dead_letter (DLQ)


RAG Service Publisher
      │
      └──► feedback_results_queue ──► TAP LMS Consumer
               (durable, persistent)
               │
               └──► feedback_results_queue_dead_letter (DLQ)
                         │
                         └──► feedback_results_queue_dlx (Exchange)

3.4 RAG Service (Frappe Custom App)

Technology: Frappe Framework + LangChain + Vision LLMs

Git Repo : https://github.com/theapprenticeproject/rag_service

Role: AI-powered analysis worker

Responsibilities:

Consume submission messages from RabbitMQ
Fetch assignment context from TAP LMS
Perform vision-based AI analysis
Generate structured feedback
Publish results back to feedback queue

Core Components:

RabbitMQConsumer (rag_service/utils/rabbitmq_consumer.py)
- Establishes RabbitMQ connection
- Declares and binds to queues
- Consumes messages from plagiarism_results_queue
- Routes to FeedbackHandler for processing
- Handles acknowledgments and rejections
FeedbackHandler (rag_service/handlers/feedback_handler.py)
- Orchestrates feedback generation workflow
- Creates/updates Feedback Request documents
- Coordinates between managers (LangChain, Assignment Context, Feedback Processor)
- Error handling and retry logic
LangChainManager (rag_service/core/langchain_manager.py)
- Manages LLM provider instances
- Constructs prompts from templates
- Formats messages for vision models
- Calls LLM APIs (async)
- Parses and validates JSON responses
- Implements fallback strategies for invalid responses
LLM Providers (rag_service/core/llm_providers.py)
- BaseLLMInterface: Abstract base class
- OpenAIProvider: OpenAI GPT-4V integration
- TogetherAIProvider: Together AI integration (Llama 3.2 90B Vision)
- Provider-specific message formatting
- Async HTTP client for API calls
AssignmentContextManager (rag_service/core/assignment_context_manager.py)
- Fetches assignment context from TAP LMS API
- Caches context in Assignment Context DocType
- Validates cache freshness
- Handles API failures with fallback to cache
FeedbackProcessor (rag_service/core/feedback_processor.py)
- Stores feedback in Feedback Request
- Formats feedback for display
- Updates document status
- Tracks completion timestamps
QueueManager (rag_service/utils/queue_manager.py)
- Publishes feedback to feedback_results_queue
- Manages RabbitMQ connections
- Message serialization
- Error handling for publishing

Technology Stack:

LangChain: LLM orchestration framework (though mostly custom implementation)
Vision LLMs: - OpenAI GPT-4 Vision Preview, Together AI Llama 3.2 90B Vision Instruct Turbo
Async Processing: asyncio, aiohttp for concurrent operations
Message Queue: pika library for RabbitMQ integration
HTTP Client: httpx for TAP LMS API calls

Error Handling:

Message Processing Errors
- Invalid JSON: Reject message (basic_reject, requeue=False)
- Missing fields: Reject message
- Processing failures: Log error, retry with basic_nack(requeue=True)
LLM Errors
- API failures: Retry with exponential backoff
- Invalid responses: Use fallback feedback format
- JSON parsing errors: Generate error feedback with standardized message
Assignment Context Errors
- API unreachable: Use cached context if available
- Missing assignment: Reject message with error log
Dead Letter Handling
- Max retries exceeded: Move to DLQ
- Permanent failures: Move to DLQ for manual review

Scalability:

Horizontal scaling: Multiple RAG Service workers can consume from same queue
QoS prefetch_count=1: Fair distribution across workers
Async/await: Concurrent processing within single worker
Connection pooling: Reuse HTTP connections to TAP LMS

4. Integration Contracts

4.1 Glific → TAP LMS (HTTP Webhook)

4.1.1 Webhook: Assignment Submission

Endpoint: POST /api/method/tap_lms.imgana.submission.submit_artwork

Authentication: API Key-based

Request Parameters (Form-encoded):

api_key: <TAP_LMS_API_KEY>
assign_id: <assignment_identifier>
student_id: <glific_contact_id>
img_url: <image_url_from_glific>

Success Response (200 OK):

{ "message": "Submission received",  "submission_id": "IMSUB-25110600001"}

Error Responses:

// Invalid API Key{  "exc_type": "ValidationError",  "exception": "Invalid API key"}// Missing Parameters{  "exc_type": "ValidationError",
  "exception": "Missing required parameter: img_url"}

Side Effects:

Creates Submission document
Publishes message to RabbitMQ queue
Commits database transaction

4.2 TAP LMS → RabbitMQ (AMQP Publisher)

4.2.1 Message: Submission Request

Queue: [submission_queue]

Exchange: ’’ (default direct exchange)

Message Body:

{  "submission_id": "IMSUB-25110600001",  "assign_id": "mathematics-week5-basic",  "student_id": "12345",  "img_url": "https://glific-storage.s3.amazonaws.com/submissions/image.jpg"}

Field Descriptions:

[submission_id]: Submission document name (auto-generated)
[assign_id]: Assignment name from Assignment DocType
[student_id]: Glific contact ID
[img_url]: Publicly accessible image URL

Queue Declaration:

channel.queue_declare(
    queue='submission_queue',
    durable=True  # Survive broker restarts)

4.3 RAG Service → TAP LMS (HTTP API)

4.3.1 API: Get Assignment Context

Endpoint: GET /api/method/tap_lms.imgana.submission.get_assignment_context

Authentication: API Key + Secret (via RAG Settings)

Request Parameters:

assignment_id: <assignment_name>
student_id: <glific_contact_id> (optional)

Success Response (200 OK):

{  "message": {    "assignment": {      "name": "Mathematics Week 5",      "description": "Solve the quadratic equation problems",      "type": "Written",      "subject": "Mathematics",      "submission_guidelines": "Show all work and steps",      "reference_image": "/files/math_reference.jpg",      "max_score": "100"    },    "learning_objectives": [      {        "objective": "LO-MATH-001",        "description": "Apply quadratic formula correctly"      },      {        "objective": "LO-MATH-002",        "description": "Show intermediate calculation steps"      }    ],    "student": {      "grade": "10",      "level": "Intermediate",      "language": "English"    },    "feedback_prompt": "Focus on problem-solving approach and accuracy"  }}

Error Responses:

// Assignment Not Found{  "exc_type": "DoesNotExistError",  "exception": "Assignment not found"}// Unauthorized{  "exc_type": "AuthenticationError",  "exception": "Invalid API credentials"}

Caching Strategy:

Response cached in Assignment Context DocType
Cache validity: configurable (default 7 days)
Cache invalidated on assignment update

4.4 RAG Service → RabbitMQ (AMQP Publisher)

4.4.1 Message: Feedback Result

Queue: [feedback_results_queue]

Exchange: ’’ (default)

Routing Key: [feedback_results_queue]

Message Properties:

properties = pika.BasicProperties(
    delivery_mode=2,  # Persistent    content_type='application/json')

Message Body:

{  "submission_id": "IMSUB-25110600001",  "student_id": "12345",  "feedback": {    "overall_feedback": "Great work on this assignment! Your approach to solving the quadratic equation was methodical and accurate. You demonstrated a clear understanding of the quadratic formula and applied it correctly.",    "strengths": [      "Clear problem identification",      "Correct application of quadratic formula",      "Well-organized solution steps",      "Accurate final answer"    ],    "areas_for_improvement": [      "Consider adding units to numerical answers",      "Double-check arithmetic in complex calculations"    ],    "learning_objectives_feedback": [      "LO-MATH-001: Excellent - Applied quadratic formula correctly with proper substitution",      "LO-MATH-002: Good - Showed most intermediate steps, but step 3 could be more detailed"    ],    "grade_recommendation": "92",    "encouragement": "Keep up the excellent work! With attention to small details like units, you'll achieve even better results."  },  "sent_at": "2025-11-06T10:35:42.123Z",  "service": "RAG"}

Field Descriptions:

[submission_id]: Reference to Submission
[student_id]: Glific contact ID for notification routing
[feedback]: Structured feedback object
[overall_feedback]: Main summary (sent to student via WhatsApp)
[strengths]: List of positive aspects
[areas_for_improvement]: Constructive criticism
[learning_objectives_feedback]: Per-objective assessment
[grade_recommendation]: Suggested numerical grade
[encouragement]: Motivational closing message
[sent_at]: ISO 8601 timestamp
[service]: Source identifier (“RAG”)

Queue Declaration:

channel.queue_declare(
    queue='feedback_results_queue',
    durable=True,
    arguments={
        'x-dead-letter-exchange': 'feedback_results_queue_dlx',
        'x-dead-letter-routing-key': 'feedback_results_queue'    }
)

4.5 TAP LMS → Glific (HTTP API)

4.5.1 API: Start Feedback Flow

Endpoint: POST <https://api.tap.glific.com/api/> (GraphQL)

Authentication: Bearer Token (from Glific Settings)

Request Headers:

Content-Type: application/json
Authorization: Bearer <glific_access_token>

GraphQL Mutation:

mutation StartContactFlow($flow_id: ID!, $contact_id: ID!, $default_results: Json) {
  startContactFlow(
    flowId: $flow_id,
    contactId: $contact_id,
    defaultResults: $default_results  ) {
    success
    errors {
      key
      message
    }
  }
}

Variables:

{  "flow_id": "123",  "contact_id": "12345",  "default_results": {    "submission_id": "IMSUB-25110600001",    "feedback": "Great work on this assignment! Your approach to solving the quadratic equation was methodical and accurate..."  }}

Success Response:

{  "data": {    "startContactFlow": {      "success": true,      "errors": null    }  }}

Error Response:

{  "data": {    "startContactFlow": {      "success": false,      "errors": [        {          "key": "flow_id",          "message": "Flow not found"        }      ]    }  }}

Flow Configuration:

- Flow must exist in Glific with label “feedback”

- Flow ID stored in Glific Flow DocType

- Flow uses [default_results] variables in message templates

- Example flow message: “Here’s your feedback for {{submission_id}}: {{feedback}}”

5. Future Enhancements

5.1 Planned v2 Features

Neo4j Knowledge Graph
- Student learning trajectory tracking
- Concept mastery over time
- Personalized recommendations
Multi-modal Submissions
- Video assignments
- Audio submissions
- Document (PDF) analysis
Real-time Progress Tracking
- WebSocket updates for submission status
- Live feedback preview
Enhanced Analytics
- Assignment difficulty calibration
- Cohort performance comparisons
- Teacher intervention triggers
Advanced Prompt Engineering
- Few-shot learning with example submissions
- Chain-of-thought prompting
- Multi-step reasoning validation

5.2 Technical Debt

Object Storage Migration
- Current: Glific-hosted URLs
- Target: S3/GCS with lifecycle policies
Plagiarism Detection
- Queue named “plagiarism_results” but feature not implemented
- Integrate plagiarism checking service
Caching Layer
- Add Redis for assignment context caching
- Reduce TAP LMS API load
Observability
- Distributed tracing (OpenTelemetry)
- Structured logging
- Custom metrics dashboards

6. Appendices

11.1 Message Flow Diagram

Student → WhatsApp → Glific
                        │
                        ▼ HTTP Webhook
                    TAP LMS
                        │
                        ├─ Create ImgSubmission
                        │
                        ▼ AMQP Publish
                   plagiarism_results_queue
                        │
                        ▼ AMQP Consume
                   RAG Service
                        │
                        ├─ Fetch Context (HTTP)
                        ├─ Call LLM API
                        ├─ Parse Response
                        │
                        ▼ AMQP Publish
                   feedback_results_queue
                        │
                        ▼ AMQP Consume
                    TAP LMS
                        │
                        ├─ Update ImgSubmission
                        │
                        ▼ HTTP GraphQL
                     Glific
                        │
                        ▼ WhatsApp Message
                     Student

6.2 Example Prompt Template

System Prompt:

You are an experienced teacher providing feedback on student assignments. Analyze the submitted image carefully and provide constructive, encouraging feedback following the rubric and learning objectives.

IMPORTANT: You must ALWAYS respond with a valid JSON object matching this exact format:
{
  "overall_feedback": "string",
  "strengths": ["string"],
  "areas_for_improvement": ["string"],
  "learning_objectives_feedback": ["string"],
  "grade_recommendation": "string",
  "encouragement": "string"
}

If the image does not appear to be a valid assignment submission, set overall_feedback to: "Something went wrong—It looks like there's an issue from our end or your submission is incorrect! I am not able to provide feedback for your submission."

Return ONLY the JSON object, no additional text or markdown formatting.

User Prompt Template:

Assignment: {assignment_name}
Type: {assignment_type}
Subject: {subject}

Description: {description}

Learning Objectives:
{learning_objectives}

Submission Guidelines:
{submission_guidelines}

Please analyze the submitted assignment image and provide detailed feedback covering:
1. Overall assessment of the work
2. Specific strengths demonstrated
3. Areas that need improvement
4. How well each learning objective was met
5. A suggested grade out of {max_score}
6. Encouraging words to motivate the student

Focus on being constructive, specific, and supportive.

At The Apprentice Project (TAP), we believe kids need 21st century skills to navigate the 21st century world. A tech-enabled not-for-profit, TAP prepares children from disadvantaged backgrounds, enabling them to be thought leaders in their communities and beyond.

Introducing Assignment Analysis: AI-Powered Feedback for Every Student, Delivered via WhatsApp

Innovation can make personalized learning feedback simple, instant, and accessible using WhatsApp.

The Challenge: Scaling Personalized Feedback

Summary

Role in the PAL Ecosystem

Key Technologies:

1. System Overview

1.1 Purpose

1.2 Key Capabilities

1.3 Architecture Style

2. System Architecture

2.1 High-Level Architecture Diagram

2.2 Data Flow: Complete Submission-to-Feedback

3. Component Details

3.1 Glific (WhatsApp Interface)

3.2 TAP LMS (Frappe Application)

3.3 RabbitMQ (Message Broker)

3.4 RAG Service (Frappe Custom App)

4. Integration Contracts

4.1 Glific → TAP LMS (HTTP Webhook)

4.1.1 Webhook: Assignment Submission

4.2 TAP LMS → RabbitMQ (AMQP Publisher)

4.2.1 Message: Submission Request

4.3 RAG Service → TAP LMS (HTTP API)

4.3.1 API: Get Assignment Context

4.4 RAG Service → RabbitMQ (AMQP Publisher)

4.4.1 Message: Feedback Result

4.5 TAP LMS → Glific (HTTP API)

4.5.1 API: Start Feedback Flow

5. Future Enhancements

5.1 Planned v2 Features

5.2 Technical Debt

6. Appendices

11.1 Message Flow Diagram

6.2 Example Prompt Template

Recent Posts

quick Links

follow us on social!