# Medical SBA Question Generator and Moderation System ## Full Project Design and Technical Documentation --- # 1. Project Overview This project is a web application designed to support the creation, editing, moderation and export of **Single Best Answer (SBA)** medical exam questions for undergraduate medical education. The system is intended for use by **medical tutors and assessment leads**, particularly within MBChB programmes. The application uses **AI-assisted generation combined with human moderation** to allow rapid creation of high quality exam items. The system is designed so that: - AI performs **content generation** - The application controls: - workflow - structure - moderation - storage - export The system is currently developed locally using **XAMPP** and will later be be deployed to a **Linux environment using PHP and MariaDB**. --- # 2. Key Goals ## 2.1 Reduce Authoring Time Automate initial question drafting using AI. ## 2.2 Maintain Academic Control All questions must be reviewed and moderated by tutors. ## 2.3 Enforce SBA Structure Ensure consistent format and high-quality distractors. ## 2.4 Enable Variant Generation Allow transformation of existing questions into new variants. ## 2.5 Provide Moderation Workflow Each question can be: - edited - approved (Done) - rejected - regenerated ## 2.6 Support Export Export to: - RISR - Excel - Word - Blackboard Learn Ultra - Microsoft Forms --- # 3. Development Environment ## Local Development - XAMPP - Apache - PHP - MariaDB - Vanilla JavaScript - Composer - PhpSpreadsheet ## Production Target - Linux server - PHP - MariaDB --- # 4. System Architecture Browser → PHP Application → MariaDB → OpenAI API ### Key Rule The OpenAI API key is **never exposed to the browser**. All AI calls are made **server-side in PHP**. --- # 5. Session Model (No Login System) The app uses **workspace-based sessions**. ### Workflow - A `workspace_id` is created on load - Stored in browser `localStorage` - All data is tied to this workspace ### Benefits - No login required - Work persists across refresh ### Controls - Start New Session - Clear Current Work ### UI Note The system still uses `workspace_id` internally, but it is no longer shown to the user on the homepage. --- # 6. Core Tools ## 6.1 Question Generator (Implemented) Generates SBA questions from structured tutor inputs. ### Current Inputs - Specialty - Core condition (optional) - Presentation (optional) - Skill(s) (optional) - Summative/Formative - Year - Tutor comments (optional) - Number of questions ### Behaviour - If optional fields are left blank, the system can generate a mix within the chosen specialty. - If multiple skills are selected, questions are distributed across them where possible. ### Tutor Comments Tutor comments are optional author guidance used to shape generation. They are distinct from question justification. Examples: - focus on primary care - include red flags - avoid rare conditions - learning outcomes can be added here if helpful ## 6.2 Variant Generator (Partially Implemented) ### Current Workflow - Upload RISR Excel file (`.xlsx`) - Validate headers against expected template - Show upload summary - Show preview table (toggle hidden/shown) - Show selectable uploaded question list - Support checkbox multi-selection - Select all / clear selection - Show details of clicked uploaded question ### Current Capacity Designed to support selection and later variant generation for up to 100 uploaded questions. ### Planned Next Steps - Variants-per-question input - Variant generation job creation - Sequential batch variant generation - Save variants into `questions` - Reuse moderation workflow - Export approved variants --- # 7. Generation Behaviour Questions are generated **one at a time**. ### Advantages - Faster first result - Continuous workflow - Reduced API failure risk Questions appear on screen as soon as they are generated so the tutor can begin reviewing immediately. --- # 8. Moderation Workflow Each question has a status: - Draft - Done - Rejected ### Actions - Edit - Mark Done - Reject - Regenerate Only **Done** questions are exported. --- # 9. Question Card Structure Each card currently displays: - Title - Status - Tags - Summative/Formative - Year - Tutor Comments - Stem - Lead-in - Options A–E - Correct answer - Justification ### Tags - Skill - Condition - Presentation - Specialty --- # 10. Editing System The edit panel is now a **right-side fixed panel** with a visible close button. ## Editable Fields - Title - Stem - Lead-in - Option A - Option B - Option C - Option D - Option E - Correct answer - Explanation / justification ### Save Mechanism Uses: - `get_question.php` - `update_question.php` ### Current UI Behaviour - click Edit → side panel opens - click Close → panel hides - click Edit again → panel reopens --- # 11. Regeneration System ### Workflow 1. Delete question 2. Start a new single-question generation job 3. Insert replacement question ### Safety A confirmation dialog prevents accidental deletion. --- # 12. Export System ## 12.1 RISR Export (Implemented) Current export is generated as CSV in RISR template column order. ### Key Behaviour - Exports **Done** questions only - Export limited to current workspace - Warns user if there are no Done questions - Includes all RISR template columns in correct order - Uses fixed default values where required - Uses blank values where columns are currently unused ### Important Mapping Rules - `Question text` = formatted full question using stem + lead-in - `Stem` = left blank - `Comments` = tutor comments - `Examiner notes` = blank for RISR export - `Summative/Formative` = from saved question - `Year` = from saved question - `Year of Writing` = current year generated automatically - `Available answers` = exported with `;1.00` / `;0.00` - `Status` = exported as `Approved` for Done questions ## 12.2 Other Planned Export Formats - Standard Excel - Word moderation pack - Blackboard Learn Ultra - Microsoft Forms ### Word Moderation Pack Likely to include: - Question - Answers - Correct answer - Justification - Metadata - Tutor comments --- # 13. Database Design The application currently uses **MariaDB**. ## 13.1 Table: `workspaces` Purpose: session management without login. ## 13.2 Table: `generation_jobs` Tracks question generation jobs. Now includes metadata fields used during generation, including: - specialty - condition_name - presentation - learning_outcome - num_questions - questions_completed - status - skills - summative_formative - year_level - tutor_comments ## 13.3 Table: `questions` Stores generated and moderated questions. Now includes: - SBA content fields - justification - specialty - condition_name - presentation - skill - year_level - summative_formative - tutor_comments - status - source tracking fields --- # 14. File Structure ```text exam-question-generator/ ├── app/ │ ├── api/ │ └── config/ ├── public/ │ ├── index.php │ ├── generator.php │ ├── variant_generator.php │ └── assets/ │ ├── css/ │ └── js/ │ ├── generator.js │ └── variant.js ├── uploads/ ├── vendor/ ├── composer.json └── composer.lock ``` ### Key API Endpoints - `start_generation_job.php` - `generate_next_question.php` - `get_workspace_questions.php` - `get_question.php` - `update_question.php` - `update_question_status.php` - `delete_question.php` - `clear_workspace_questions.php` - `export_done_questions.php` - `upload_risr_file.php` --- # 15. AI Prompt System Two major prompts define the behaviour of the AI. - Prompt 1: SBA Generator - Prompt 2: Variant Generator ### Prompt Responsibilities - clinical accuracy - SBA structure - distractor quality - explanation clarity - variant generation rules --- # 16. Current Development Status Implemented and working: - Workspace creation and local storage persistence - Start New Session - Clear Current Work - Generation jobs - Sequential AI question generation - Real OpenAI API connection through PHP - Question cards rendered progressively - Questions saved to MariaDB - Questions reloaded after refresh - Mark Done moderation action - Reject moderation action - Regenerate with confirmation and replacement generation - Full SBA edit panel - Edit panel side panel UI - Status filters - Export readiness summary and list - RISR-aligned export - Summative/Formative capture, display and export - Year capture, display and export - Tutor comments capture, display and export - Variant Generator upload page - RISR `.xlsx` upload using PhpSpreadsheet - Header validation - Upload preview toggle - Checkbox selection list - Clicked question details panel --- # 17. Next Development Steps ## 17.1 Variant Generator Engine - Add variants-per-question control - Create variant generation jobs - Generate variants sequentially - Save variants to database - Reuse moderation flow for variants ## 17.2 Further Export Development - Real `.xlsx` export if needed - Word export - Blackboard Learn Ultra export - Microsoft Forms export ## 17.3 UI Improvements - Better layout and spacing - Optional tabs/sections for generator vs variant tools - Additional validation and user guidance --- # 18. Future Enhancements Possible future additions include: - Question quality scoring - Difficulty estimation - Duplicate detection - Curriculum coverage analysis - Question bank search - Multi-user authentication --- # 19. Long Term Vision This system is intended to become a **complete SBA authoring and moderation platform** for medical assessments. ### Core Capabilities - AI-assisted generation - Human moderation - Structured item writing - Variant generation - Multi-platform export ### Strategic Goal To significantly reduce the time required to produce high quality SBA exam questions while maintaining full academic oversight.