ECE 605: Large-scale AI
Syllabus
This course is designed to give the technical background for a deep understanding of the problems associated with the development, deployment, and exploitation of Large Language Models, providing the students with a sufficient knowledge base to perform research directed at resolving these problems in academia or industry.
Instructor:
Prof. Igor Molybog
molybog@hawaii.edu
Course website:
Pre-requisites:
Solid coding background (Python, PyTorch)
Linear algebra and statistics fundamentals
Familiarity with the topics covered in the Undergraduate version of the class
(If you did not take it or any equivalent, you can see slides, watch lecture recordings, and check out homework assignments)
Logistics
The class is split into two parts:
The lectures will start the semester by providing the necessary background to orient oneself in the vast LLM-related literature. The second part will involve the presentation and discussion of recent important papers on statistical models. The objective of the course is to instill a holistic view of the latest developments in AI, and help the participants understand their broad implications.
The class does not have homework assignments, but it has a final project.
Role-playing seminar
Each paper will be presented by a group of students, each assuming a specific “role.” These roles provide distinct perspectives and help structure your exploration and understanding of the paper. Although responsibilities are separated across individual roles, the presentation is a collaborative team activity. Below are detailed instructions and guiding questions for each role to ensure your presentations are insightful and educational.
Stakeholder ✍️ (18 min + 3 min Q&A)
Act as the authors of the paper. Your goal is to communicate the essence of your work clearly.
Motivation:
Problem Definition:
Methodology:
What specific approach did you choose, and why?
Describe your model, algorithm, or experimental setup clearly and concisely.
Experimental Findings:
Summarize your key results and highlight the most important findings.
Why do these results matter?
What insights do these results provide?
Scientific Reviewer 🔎 (8 min + 3 min Q&A)
Critically evaluate the paper as if reviewing for a prestigious conference (e.g., NeurIPS “Review Content” or ICML “Main Track Reviewer Form Instructions”). Your review should be actionable rather than descriptive. In other words, it should provide explicit instructions on improving the paper.
Empiricist 👩🏽🔬 (11 min + 3 min Q&A)
Conduct a small-scale implementation or practical exploration of the paper’s ideas or data. Empiricists should aim to reproduce a result and take a next step forward, verifying new hypotheses or extending arguments about the existing claims.
Experimental design:
Implementation:
Empirical Observations:
Challenges & Insights:
Archaeologist 🏺 (11 min + 3 min Q&A)
Archeologists should contextualize the paper within existing research literature. They should aim to put the paper into perspective, reflect on alternative ways to solve the problem, and emphasize novelty and originality.
Visionary 🔭 (11 min + 3 min Q&A)
Imagine and propose a creative yet realistically implementable follow-up research project or novel application inspired by the current paper, and explore how the ideas connect to the course project challenge.
Team Coordination Checklist (1 week prior to presentation, once all presenters have read the paper; exception: first two weeks):
The Visionary is responsible for creating a group thread with the other role presenters in the Discord forum and curating the conversation.
Coordinate so subsequent roles do not repeat background, architecture, or baseline summaries already covered by the Stakeholder.
Ensure that the Empiricist's practical exploration aligns with and complements the Reviewer's critique.
Verify that proposed follow-ups account for subsequent literature identified by the Archaeologist.
Confirm all role slides are properly titled and uploaded to the shared pool of slides at least 1 hour before class.
Connection to the Challenge:
Proposed Research Project:
New Applications:
Roles can be presented individually or in groups of up to 3. If working in groups, ensure each member has a clear, equitable role to present and discuss during class.
Who presents what role and when? Starting from the beginning of the semester, students will be able to assign themselves to a paper/role using the sign-up sheet. Please make sure to enter your name, Discord handle on the Students tab of the sign-up sheet so your classmates and the professor can coordinate with you.
There will be a line that slowly moves down the sign-up sheet as the semester progresses. Any open slots that were not filled out voluntarily will be assigned randomly. Above this line, all voluntary and randomly assigned slots are fixed:
You cannot disappear from a slot above the line (unless you drop the class).
You can appear above the line if someone else drops the class and the vacant spot gets reassigned.
Please check the sign-up sheet often to stay aware of your upcoming presentation dates.
If swapping roles/slots due to time conflicts, please discuss it in the Discord forum and notify the professor. If dropping out of the class, please notify the professor.
Each role should aim for the specified time budgets. The professor will time every presenter with a countdown clock for both the presentation and Q&A. Additional time will only be granted if there are roles that don't present. You are required to have slides for your role. I would recommend less than 7-10 slides to make sure you stay within our time budget.
What slides? To minimize time spent context switching or fighting with screen sharing/projector dongles, we will have a shared pool of slides. Each role group is encouraged to title their slides with “[role name]: [student name]” (as in “Archeologist: Jane,John”) so that the slides are quickly identified during the session.
Non-presenter role? If you aren't in the presenting group during a given class period, you are supposed to read through the paper at home and come up with a question you can ask every presenter. After a presenter is done with presenting, you will have the opportunity to ask questions. (For the first two weeks of class, please introduce yourself when asking a question so everyone gets to know each other.) Your grade for the class will heavily depend on your activity in every seminar (not just your presentations), so please make sure to speak up and be proactive.
Collaborative Q&A Culture: Q&A sessions are intended to be collaborative, constructive class discussions rather than direct interrogations of presenters. When presenters hit knowledge gaps, constructive contributions and help from the audience are highly encouraged and valuable for the entire class. They will be counted toward increasing the grade of both the presenter and those helping the presenter to answer the question.
Project
For the Final Project, students will have the opportunity to participate in a live ML challenge (2026 BEHAVIOR Challenges).
To simulate a real-life collaborative research experience, we will aim to form teams of minimum 3 and maximum 5 people each. Team selection must be finalized by the end of Friday (HST) of Week 2 of the class (using the sign-up sheet).
The project teams will have weekly consultations with the professor and write regular reports.
Team Roles and Project Management
To ensure clear coordination and accountability, each project team should define internal roles:
Coordinator: Each team must designate a Coordinator (marked with “x” in the corresponding column of the Team Mapping sheet in the sign-up spreadsheet). The Coordinator is responsible for:
Coordinating team communication and curating team discussion threads.
Scheduling internal team meetings and coordinating weekly consultations with the professor.
Submitting the collective team milestone reports (Summary Proposal, Midway Report, and Final Report).
Tracking overall team progress and timelines.
Technical Roles: Teams are encouraged to distribute specialized responsibilities across members (e.g. technical planning, data pipeline & environment setup, baseline modeling, evaluation & ablation experiments).
Cross-Team Collaboration
While each team pursues their own solution, cross-team collaboration is strongly encouraged. Sharing compute infrastructure scripts, data preprocessing pipelines, baseline code, and helping unblock peers across teams simulates real-world collaborative AI lab environments. Meaningful cross-team contributions should be highlighted in your Personal Contribution Statement and will be positively rewarded in the grading rubric.
Compute Resources
Students and project teams have several options to consider for computing and GPU resources:
UH Koa Cluster: High-Performance Computing (HPC) cluster provided by the University of Hawaiʻi. See the Koa cluster onboarding guide to get set up.
ACCESS Allocation: National Science Foundation (NSF) ACCESS supercomputing allocation. Students will need to register at the ACCESS website and indicate their ACCESS username and Discord handle on the Students tab of the sign-up sheet to gain access.
GCP Trial Credits: Google Cloud Platform provides free trial credits upon sign-up (offering limited GPU resources).
Other Paid Resources (H100 pricing reference as of Jun 2025):
Project Deliverable Guidelines
To maintain continuous momentum, accountability, and regular personal feedback, students submit Weekly Individual Progress Reports, alongside the three major team milestone reports: the Summary Proposal, Midway Report, and Final Report.
Weekly Individual Progress Report
Due: Friday by end of day (HST) every week
Submission Form: Assignment Submission Form, assignment “Weekly Individual Progress Report”
To foster accountability, each student is required to submit an individual report (PDF or Markdown file) in a concise 4-point format:
Work on class planned for this past week: What did you set out to achieve for the project this week?
Done this week: What concrete progress, research, code, or tasks did you complete?
Plans for the next week: What are your specific goals and milestones for the upcoming week?
Blockers / Issues: What bottlenecks, technical difficulties, or questions are slowing you down?
These weekly reports directly inform consultation discussions and the individual contribution score.
Summary Proposal
Due: Friday of Week 6 (Maximum: 2 pages)
Your proposal should briefly but clearly outline your project idea, approach, and goals. The recommended structure is:
Problem Definition and Motivation (1-2 paragraphs)
Clearly state the problem.
Explain why this problem is important or relevant.
Proposed Approach (1-2 paragraphs)
Outline the approach or methods you plan to use.
Concrete Goal (clear and specific)
Avoid broad statements.
Provide focused, concrete goals (e.g., “I will verify hypothesis X through experiments Y1, Y2, Y3”).
Expected Outcomes
Briefly describe the results you anticipate.
Required Resources & Technical Infrastructure
Data sources you will use.
Computational resources (GPUs, cloud resources, etc.).
Tools, software libraries, and frameworks.
Motivation & Impact
Who will benefit from your project and how?
Potential applications and real-world scenarios.
Measurable Objectives & Success Criteria
Clearly define quantifiable metrics for success.
Specific research questions your project will answer.
Milestones and Timeline
Provide a roadmap with manageable phases and realistic deadlines.
Background Research and References
Summarize key existing literature or projects related to your work.
Include relevant references.
Training Objectives and Skills Development
List the technical and professional skills you aim to enhance.
Midway Report
Due: Friday of Week 11 (Maximum: 5 pages)
The midway report documents your progress and provides a clear plan for the remaining project work.
Your midway report must include:
Current Progress
Clearly describe what you have completed.
Document preliminary experiments and results.
Data Details
Raw data format and source.
Steps taken in data preprocessing.
Data Pipeline
Explain your data processing pipeline clearly.
Benchmarks and Metrics
Clearly define how you evaluate the performance of your solution.
Baseline and Methodology
Provide a clear description of baseline methods for comparison.
Describe your proposed methodology in detail.
Ablation Studies
Describe experiments to isolate and understand different components of your solution.
Results
Clearly present initial results.
Plan for Remaining Work
Detail your remaining tasks and timeline.
Final Report
Due: Friday of the last week of classes (Maximum: 8 pages, excluding references)
Your final report should emulate the structure of a standard conference paper. It must comprehensively summarize your project from concept to conclusion, clearly communicating your findings.
Include the following sections:
Abstract
Concisely summarize your problem, approach, results, and conclusions.
Introduction
Clearly state the problem and motivate why it is important.
Summarize relevant literature, technologies, or prior solutions.
Methodology
Explain the approach you chose, justifying why this method is suitable.
Include a clear description of the data pipeline, model architectures, algorithms, and other relevant technical details.
Experiments and Evaluation
Clearly describe experiments conducted.
Provide benchmarks, metrics, and evaluation methodologies.
Results and Discussion
Present and interpret your results clearly.
Discuss what worked, what did not, and why.
Conclusion and Future Work
Summarize findings and their implications.
Suggest directions for future work.
Contributions (for group projects)
Explicitly list each member's contributions.
References
Properly cite all relevant sources.
Example project reports.
A couple of years back, the project topic was more open-ended. Based on the feedback, we introduce more structure to the project’s goals. However, the format of milestone reports remains largely unchanged.
Examples in the table below are aiming to provide a feeling of how a report and feedback might look. The reports are not perfect, and some criticism is provided in the feedback section.
Please feel free to use the template for your project report.
Personal contribution statement
Aside from the collective team report (submitted by the Coordinator), there is a required submission of a personal contribution statement (one per team member). This way, the Coordinator submits 2 reports per milestone (team + personal), while the rest of the team submit one report per milestone (personal contribution). This requirement is put in place to mimic the (semi-)annual review process common in industrial AI labs.
In your personal contribution statement, clearly highlight:
Your direct technical deliverables and ownership within your team.
Cross-team collaboration & enablement: Any tools, shared infrastructure, data pipelines, code reviews, or troubleshooting assistance you provided to help other teams in the class.
Please feel free to use the Personal Contribution Statement (template) to format your report (optional).
Project reports should be submitted through the form: here
Grade calculation
Role-playing seminar participation grade is calculated roughly using the following rule:
0 points for just coming to class and not participating
1 point for coming to class and participating (actively), but not presenting
2 points for presenting (well)
-1 point for not showing up when you are not presenting
-2 points for not presenting on schedule (whether you showed up or not)
3 points for top performance of the day (usually among standout presenters or active discussion contributors)
Collaborative Q&A in Grading:
For Presenters: Answering questions thoroughly demonstrates command of the paper and secures top presentation marks (+2 to +3 points). If you encounter a knowledge gap or difficult question, engaging openly and inviting/receiving helpful clarifications from the class still adds a plus to your performance (though directly resolving the question yourself earns the most credit).
For Audience Members: Stepping in constructively to help answer questions, share insights, or bridge knowledge gaps during Q&A is highly valued and counts directly toward active participation (+1 or +3 points).
Presentation Feedback: Detailed feedback on your presentation is available through 1:1 conversations with the professor. Please book office hours to review your performance and get actionable feedback.
Project performance is calculated in two steps, roughly using the following rule:
Step 1 (Team Performance): The team’s work at the milestone is evaluated on technical quality, progress against goals, rigor of experiments, and reproducibility, resulting in a Team Performance Score (0–100).
Step 2 (Individual Contribution Weighting): that score is then allocated across team members in proportion to demonstrated contribution. Contributions are inferred from weekly progress reports, weekly consultations, milestone artifacts (code, experiments, docs), and each student’s Personal Contribution Statement, with light peer input for calibration. As a guardrail, weights are usually bounded to a reasonable range (e.g., ~0.7×–1.3×) unless there is clear evidence for outliers. The default assumption is equal shares when evidence is insufficient. Students may request a brief calibration check within one week of feedback if they believe their weighting doesn’t match the submitted evidence.
Rubric for evaluating personal contributions:
Technical ownership & delivery (40%): scoped, shipped, reproducible artifacts tied to roadmap. Includes technical enablement of other students in the class (teammates or other teams)
Measured impact (25%): metrics moved vs. baseline; ablations; clarity of results
Collaboration & enablement (20%): intra-team and cross-team collaboration, shared infra, code reviews, docs, unblocking other students
Rigor & safety (10%): data hygiene, eval soundness, reproducibility, risk checks
Communication (5%): crisp reports, decision logs, responsible self-assessment
Class grade =
0.6 * Participation (presentation + discussion participation) +
0.4 * Project (team performance * personal contribution)
Evaluation of the Course
Students’ continued feedback is highly appreciated, both positive and negative. To that aim, a form will remain open throughout the semester where you can provide feedback directly to me while remaining anonymous.
You can submit feedback here. I will read every submission and take it into consideration.
There will also be an opportunity to provide formal feedback and evaluate the course at the end of the semester.
Course Policies
Attendance & Sign-in
To record attendance and track participation for grading, an in-person physical sign-up sheet will be available before class starts each session. Please ensure you sign the sheet when you arrive.
AI
This course is on AI systems; you are welcome (and encouraged) to use AI tools for preparation for your presentation. Examples: Chatbots for answering questions on the paper; coding agents for empirical experiments, Claude for slide drafting. etc. The grade will prioritize the quality of the defence of the presentation and the activity in the discussion. Thus, it is critical that you understand the material, even if you used AI for some part of the preparation.
Inclusion
We are committed to creating a learning environment welcoming of all students that supports a diversity of thoughts, perspectives and experiences, and respects your identities and backgrounds (including race/ethnicity, nationality, gender identity, socioeconomic class, sexual orientation, language, religion, ability, etc.) To help accomplish this:
Please let us know if you have a name and/or set of pronouns that differ from those that appear in your official records.
If you feel like your performance in the class is being impacted by your experiences outside of class (e.g., family matters, current events), please do not hesitate to come and talk with us. We want to be resources for you.
We (like many people) are still in the process of learning about diverse perspectives and identities. If something was said in class (by anyone) that made you feel uncomfortable, please talk to us about it.
As a participant in this class, recognize that you can be proactive about making other students feel included and respected.
|