Skip to content

Add LLM-based discussion to async peer instruction (course-wide key)#1140

Merged
bnmnetp merged 23 commits intoRunestoneInteractive:mainfrom
sethbern:main
Feb 10, 2026
Merged

Add LLM-based discussion to async peer instruction (course-wide key)#1140
bnmnetp merged 23 commits intoRunestoneInteractive:mainfrom
sethbern:main

Conversation

@sethbern
Copy link
Contributor

@sethbern sethbern commented Feb 6, 2026

This adds an LLM-based discussion mode to asynchronous peer instruction, using a course-wide OpenAI API key when available and falling back to legacy behavior of showing previous text-chat messages when there is no key.

Core functionality

  • Adds an LLM discussion phase between Vote 1 and Vote 2 in async peer instruction
  • Uses a course-wide OpenAI API token (via fetch_api_token)
  • Automatically falls back to legacy async peer instruction when no API key is present

Implementation details
Backend:

  • _llm_enabled() checks for a course-wide OpenAI token
  • _call_openai() performs a minimal HTTP call using that token
  • get_async_llm_reflection():
    • Logs student reflections and messages
    • provides MCQ context (question, code, choices)
    • generates a peer-style LLM response
    • All failures (missing key, API errors) fail fall back to legacy behavior

Frontend:

  • Ensures proper flow for vote 2
  • Requires at least one LLM message before allowing vote 2

How to use:

  1. Add a course-wide OpenAI API token
  2. Async peer instruction automatically enables LLM discussion after second vote

@sethbern sethbern requested a review from bnmnetp as a code owner February 6, 2026 20:10
@bnmnetp
Copy link
Member

bnmnetp commented Feb 6, 2026

@sethbern All the code under bases/rsptx/web2py_server is deprecated. I have repeatedly asked (not you directly) that new features be done on the FastAPI based servers. I sent the following to your colleague Ashlyn earlier this week.

I would recommend that you look at https://docs.runestone.academy and at least have a look at the welcome and overview of the monorepo that describes how things are structured. There is lots of other documentation on building, developing and debugging there as well.
The PI code has been ported to the assignment server.

Hence it is going to live under rs/bases/rsptx/assignment_server_api. There is a folder there for routers and that contains peer.py. The Jinja templates for the pages are in components/rsptx/templates/assignment (either under student or instructor)
The peer.js file is in components/rsptx/templates/staticAssets/js

All the code needed to retrieve and update from the database is under components/rsptx/db/crud If you need to add anything there please take note of how things have been added before and follow those patterns.

Please be careful not to confuse these files with the old files that are still present under bases/rsptx/web2py_server. That is the old version that is currently in production but we are phasing that out. The new code has been tested but definitely not to the extent that the old code has been.

Feel free to reach out for help in our Discord, or also feel free to come to our tuesday zoom dropin ( https://mathtech.org/dropin) I’m usually there from 8-10 Pacific time. But if I’m not others will be there that can answer questions too.

Its fine if we want to have these features in parallel, but the plan is that the web2py service will be deactivated next summer.

content Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

content, docker-compose.override.yml and pi_attempt_id all appear to be empty files. Please remove them from the PR.

@sethbern
Copy link
Contributor Author

sethbern commented Feb 7, 2026

@bnmnetp Thank you for your comments. I have removed the empty files. I understand that this work should ultimately live in the FastAPI assignment server and that the web2py code is being phased out. However, I want to check, would you like me to move this over to FastAPI now, or is it okay to merge the web2py version and then follow up with the FastAPI version right away?

@bnmnetp
Copy link
Member

bnmnetp commented Feb 7, 2026

Its OK to leave what you have in place for web2py, but I would encourage you to get it over to FastAPI and use that version for your research. That PI that I have ported will be available (though not the default) in production this weekend.

Whether you do that as one PR or two is not a big deal to me.

@sethbern
Copy link
Contributor Author

sethbern commented Feb 7, 2026

@bnmnetp Thanks, I appreciate the clarification.

I’ll keep this PR focused on the existing web2py implementation so it can be merged and used as is for an upcoming conference workshop. I’ll then follow up with a separate PR that migrates the full functionality over to the FastAPI assignment server and uses that version for the research going forward.

@bnmnetp bnmnetp merged commit 801a379 into RunestoneInteractive:main Feb 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants