(###)

CPSC 538M: Topics in Systems Security

The goal of the project is to provide the opportunity for you to conduct research in some security-oriented topic.

Projects overlapping with other research: You could undertake a project related to your own research, if you can demonstrate how it is related to and/or influenced by some topic from CPSC 538M. In other words, your project in CPSC 538M must extend your research work in some new and/or different way.

Deliverables

In the project, you must propose a research project with a problem statment and a research plan, conduct the research, and write up your research results and experience. There are four deliverables in the project: a research proposal, a proposal presentation, a final presentation, and a final report.

Note: It is okay if you do not complete a full-fledged project by the end of the term. The goal is to learn how to go from a one-line problem to a fully scoped out research problem, then try and identify potential solutions. If a topic is difficult and you do not reach the practical implementation stage, that will be fine.

Abstract (5%)

Context: A brief background on the domain your problem is in, and why this is an important domain today. A brief background on the problem in the domain that are you planning to address, and why it is an important problem to solve.

Gap: Why is the problem difficult to solve? Why have prior techniques failed?

Innovation: What is the key idea of your solution? What is novel about the solution?

Impact: How does your solution improve the state of the art? How does your solution help the target domain (e.g., performance, usability, revenue, avoid lawsuits)?

Each project type, including exploration, replication, and systematization, can have a similar structure for the abstract as well as the subsequent proposal and final reports. For instance, for replication and systematization studies, explain the need for the studies, the challenges involved, and the expected results and insights at the end of the studies.

Research proposal (15%)

The proposal should include the following sections and must be minimally three pages, excluding bibliography, typeset in two-column format. Use the USENIX style template.

Introduction: In this section, you will explain what is the problem, why is it an important problem, and why are you interested in this problem? Describe the background and motivation for the problem you are interested in, and pose a concrete research question. The extended abstract could serve as the basis for this section.

Background and related work: Do a brief survey of related work in the problem space. This includes papers that solve the same problem but with a different approach and papers whose ideas you build upon in your own work. Compare and contrast your own proposed solution with each related work. Some of the papers in related work may be required to explain the background and motivation in the introduction.

Even if you have not read all the related work by the time of proposal submission, make a list of papers that you will be covering by the final report submission.

Challenges: Describe the technical challenges that must be addressed to solve your problem. You can think of this as a breakdown of your big problem into a series of small questions whose answers together help to address the big problem.

Proposed solution: Describe your proposed solution and planned methodology to answer the research question at a high level. Also, describe the steps to address the small questions from the previous section that will help you get to the overall solution.

Evaluation plan: Describe what kind of artifact (if any) you plan to develop and how you plan to evaluate your proposed solution. What is your security evaluation plan? What is your performance evaluation plan? What kind of data (or plots) would you generate from the evaluation?

Experimental setup: What equipment, software, tools do you need for your solution and evaluation? If you need anything specific, please speak to us as soon as possible.

Timeline and deliverables: Provide a breakdown of your project plan into a few small milestones with deliverables at each stage. This breakdown will guide you through the project and provide an objective evaluation of your progress.

Relation to other research: What is the relation of the project with your other research (if applicable)?

Proposal presentation (10%)

Each group will give a 5-7 min presentation followed by 3-5 min of Q&A and feedback from the class. Focus on presenting:

• Problem: What is the problem you are working on? (Include the basic context that is required to understand the problem statement.)
• Motivation: Why is it an important problem and why did you choose to work on this problem?
• Key idea: What is the key idea of your solution?
• Deliverable and Timeline: What is the expected final deliverable and how you plan to get there??

Final presentation (20%)

Each group will give a 7 min presentation followed by 5 min of Q&A from the class. The presentation would be similar to a conference or workshop talk and would include the following aspects.

• Problem: A brief description of the problem and the motivation.
• Challenges: A list of challenges that need to be solved, highlighting which challenges you focused on solving in your project.
• Key idea: A longer discussion of one key idea of your project addressing one of the challenges you solved.
• Evaluation goals: A list of your evaluation goals, indicating what to measure and how to evaluate the success of your project. This may be quantitative metrics (e.g., performance improvements, or number of new vulnerabilities found) or qualitative (e.g., anticipated reduction in human effort by using a given tool).
• One cool result: A discussion of one of the results you may have achieved in the project.
• Limitations and future work: A brief discussion on limitations of the current work and potential directions for future work.

Final report (50%)

The final report must include similar sections as in the proposal but written in a way to describe what has been done. It should be minimally 5 pages, excluding bibliography and appendix, typeset in two-column format. You can continue from the proposal. Expected sections (feel free to adapt section titles and structure):

• Abstract: Max 4-5 sentences describing context, gap, innovation, results, and impact. You might want to write this at the end, after the rest of the report is completed.
• Introduction: Extended version of abstract, which describes the context and the problem, why it is an important problem, why it has not been solved yet (which may include a high-level discussion of the limitations of prior work that attempted to solve the problem and/or the challenges that make the problem difficult to solve), the concrete question solved in your work, the key ideas of your solution, a brief overview of your evaluation methodology and results, and a list of concrete contributions.
• Background and related work: By now, you would have studied more papers relevant to your project. A well-written background and related work section serves as education material for a non-expert reader. The background should describe any work that forms the basis of your own work. It serves as a building block for the readers to understand the core sections of your work. If your work involves math or formal modeling, describe any definitions, axioms, theorems, or similar building blocks that a reader must know. If your work involves dependency on specific hardware, architecture, a previously designed system, specific tools or methodology, describe its salient features. The goal of the related work is to convince a reader that your work is novel and advances the state of the art in the problem space that you investigated. It should describe any work that solves a similar problem with a different methodology than yours, a different problem but with a similar methodology as yours, or a similar problem with a similar methodology (which would be a very closely related work). For each type of work, briefly describe the work, the similarities and differences between your work and the related work (including pros and cons of both), and whether your work could have been building on the prior work (if appropriate).
• Threat Model/Challenges/Design Goals: Describe the threat model and the technical challenges or goals that your project aims to solve. The threat model should describe the victim and its target use cases, what is to be protected, the adversary, its position, how it gets there and what it can observe and perform, the trusted computing base, the threats that are out of scope, and a brief justification of why this is a realistic and important threat model. This challenges or goals may include points related to security, performance, usability, modularity and portability, energy efficiency, etc.
• Solution Overview: Describe the design of your solution at a conceptual level, without getting into implementation details. Where possible, describe the ideas with the help of a running toy example that you might refer to throughout the section. The example may be a part of your later evaluation. Use figures to illustrate the ideas.
• Implementation: Describe the components that constitute your solution, how they work and integrate into an existing system (if applicable). For the components you added, describe the size of the code you added or modified, the language of implementation, the memory footprint or any other relevant metrics to explain the effort and cost of building the solution. Provide a link to your artifact (e.g., on github, or any other url).
• Evaluation: List the concrete goals of evaluation and what it aims to demonstrate. Describe the experimental setup, datasets, and workloads used. For each experiment, describe the methodology, i.e., how you ran the workload and the metrics you measured, report the numbers (use tables, plots, or text appropriately), an interpretation and an explanation of the numbers, and a conclusion or takeaway from the observations.
• Limitations and Future Work: Describe what needs to be accomplished (by you or someone else) to complete the current scope of the project and how the research can be extended beyond its current scope in the future.
• Conclusion: A two-line summary of the project and a brief description of the key takeaways for a reader.
• Appendix: The timelines from proposal stage and the second check-in stage, as well as a brief description of the challenges encountered during the project, a reflection on how the timeline evolved, and what could you do differently in future.

Feedback

Each student will be required to give feedback on at least one other group’s project proposal. Your feedback will count towards your class participation points. We will use HotCRP to simulate a conference-style review process.

In addition to feedback from the class, each group can schedule meetings with the instructor to discuss their project idea, methodology, milestones, and progress. At the minimum, we expect to meet with you once before the proposal submission deadline and once before the final presentation. You can schedule more meetings with us on an on-demand basis.

Project Milestones

• Sep 11: Register team: Post your team information on Piazza.
• Sep 25: Abstract due: Submit an abstract for the project (max 1 page).
• Oct 01: 1st check-in: Schedule a 20min 1-1 meeting with me between 12-2pm to discuss project scope.
• Oct 07: Proposal due: Submit project proposal.
• Oct 14: Peer review: Submit feedback for two other proposals on HotCRP.
• Oct 21: Proposal presentation: Present project proposal in class.
• Nov 05: 2nd check-in: Schedule a 20min 1-1 meeting with me between 12-2pm to discuss project status.
• Nov 27: Final presentation: Present project results in class.
• Dec 04: Final reports due: Submit final course project report.