3-2 Final Project Milestone One: Study Selection and Summary

The goal of the project is to provide an opportunity to gain in-depth experience developing an AI-based approach to a healthcare problem. Since a large part of the course is focused on deep learning approaches for biomedical data, we require that your project must involve training of at least one deep learning model on biomedical or more broadly healthcare-related data. To satisfy this requirement, simply re-running training of a model from an existing code repository is not sufficient; you must at minimum demonstrate the ability to perform thoughtful monitoring and analysis of the training process and hyperparameter selection. Beyond this requirement, the precise topic of your project is flexible to allow exploration of diverse interests.

ORDER THIS PAPER HERE

You may work in groups of 1-3 on the project, and grades will be calibrated by group size. Your project may be related to research in another class project as long as permission is granted by instructors of both classes; however, you must clearly indicate in the project proposal, milestone, and final reports the exact portion of the project that is being counted for this course. In this case, you must prepare separate reports for each course, and submit your final report for the other course as well.

Following is a description of the project components that you will need to turn in, the grading rubric, as well as some suggestions for project ideas.

Project Proposal

Your project proposal is due on Friday, Oct 9. It should be a 1-1.5 page document excluding references and figures, using the NeurIPS template. The proposal should include the following (with grade breakdown):

  • Title, Authors(s)
  • (25%) What is the problem that you will be investigating? Explain the task thoroughly. Why is it interesting?
  • (25%) What is the data you will be using? Please include relevant characteristics such as the source of the data, the size of the dataset, and a sample of the data. Explain clearly which parts of the data will be used as well as potential obstacles.
  • (25%) What methods do you plan to experiment with? Please thoroughly describe them. If there are existing related implementations, will you use them and how? How do you plan to improve or modify such implementations? This can be subject to change, but you should have a general sense of how you plan to approach the problem you are working on.
  • (25%) How will you evaluate your results? Qualitatively, what kind of results do you expect (e.g. plots or figures)? Quantitatively, what kind of analysis will you use to evaluate and/or compare your results (e.g. what performance metrics or statistical tests)? What is your hypothesis regarding your results compared to baselines?

Submission: Please submit your proposal as a PDF on Gradescope. Only one person on your team should submit. Please have this person add the rest of your team as collaborators as a “Group Submission”.

Project Milestone

Your milestone is due on Friday, Oct 30. It should be a 3-4 page document using the NeurIPS template. It should include preliminary sections towards the final report, in standard formats of NeurIPS research papers. The milestone should include the following (with grade breakdown):

  • Title, Authors(s)
  • (20%) Introduction. Introduce your problem, and the landscape for why the problem is interesting and what has been done before in this space. Describe your overall plan for approaching the problem, what contributions you expect to make, and why this is interesting in the context of the described landscape.
  • (25%) Related Work. Describe in detail existing work related to your problem, how they are related to each other, and how your work relates to these. We expect this to be comprehensive and thorough, and with at least 10 citations discussed and cited accordingly.
  • (15%) Data. Describe in detail the data that you are using, including the source(s) of the data, relevant statistics, and qualitative examples if appropriate.
  • (20%) Approach. Thoroughly describe the methods that you intended to use in your approach, and the baselines you plan to compare against.
  • (20%) Preliminary Results. Describe any preliminary results up to the time of the milestone. You should show the results of training at least one deep learning model. (It is fine if the model has not yet attained good performance.) You should also show preliminary analysis on the model(s) that you have trained. You should also describe anticipated next steps and any obstacles that have come up.

Each of these will be graded on a scale of 0-3 based on how well each component of each section is addressed. You can look at past NeurIPS papers for examples of how to compose each of these sections.

Submission: Please submit your proposal as a PDF on Gradescope. Only one person on your team should submit. Please have this person add the rest of your team as collaborators as a “Group Submission”.

Project Midpoint Presentation

Your project milestone presentation is scheduled on Monday, Nov 2. Project groups will present their milestone work in a 4-5 minute presentation during the lecture slot of Nov 2. Each group will have the choice to present live or record a video. More details will be provided closer to the date.

Project Advising Session with TAs

Between the project milestone and the project final report / presentation, we will schedule an advising session for each project group with the TAs, to discuss and receive additional feedback. More details will be provided closer to the date.

Project Final Presentation

We will be holding a virtual final presentation to share the work that you’ve done in your projects, during the final lecture slot of the course. Each project group should present a 4-5 minute slidedeck during the session and will have a choice to present live or record a video. More details will be provided closer to the date.

Project Report

Your project report is due on Wednesday, Nov 18. It should be a 7-9 page document using the NeurIPS template. The report should include the following sections (with grade breakdown):

  • Title, Authors(s)
  • Abstract. A paragraph overview of the problem, approach, contribution, and key results.
  • (15%) Introduction. Introduce your problem, and the landscape for why the problem is interesting and what has been done before in this space. Describe your overall plan for approaching the problem, why this is an interesting contribution in the context of the described landscape, and a summary of your results.
  • (15%) Related Work. Describe in detail existing work related to your problem, how they are related to each other, and how your work relates to these. We expect this to be comprehensive and thorough, and with at least 10 citations discussed and cited accordingly.
  • (15%) Data. Describe in detail the data that you are using, including the source(s) of the data, relevant statistics, and qualitative examples if appropriate.
  • (20%) Approach. Describe in detail the methods that you use in your approach. Importantly, through this section and the experiments section (which may include implementation details), there should be sufficient information for others to reproduce your results.
  • (25%) Experiments. Describe experiments that you performed to support your approach and contribution. The exact experiments may vary depending on the project, but we will be looking for the thoughtfulness of your analysis. Examples may include performing comparison of your main approach with other baselines or methods, error analyses to investigate the performance of the model, ablation studies to determine the impact of various components of the approach, analyses to provide insight into the effect of different hyperparameter choices, techniques to interpret how the model is working, etc. You should include graphs, tables, or other figures to illustrate the experimental results.
  • (5%) Conclusion. Summarize the key results, what has been learned, and avenues for future work.
  • (5%) Writing/Formatting. Your paper should be clearly written and nicely formatted, comparable to published NeurIPS papers.
  • Contributions. Specify the contributions of each author on the paper. This includes discussion, implementation, and writing for each part of the paper. You should also describe the contributions of any contributors not enrolled in the course (see Additional Submission Requirements below). For an example of appropriate format, please see the author contributions for AlphaGo (Nature, 2016). However, we expect your description to include the more detailed breakdown specified here.
  • Supplementary Material. This should be submitted as a separate file from your paper and is not counted in the 7-9 page requirement. At minimum, this should include the relevant code for your project. You may also put additional visualizations, demos, videos, etc. that you wish to share with the teaching team.
    What you should not put in your supplementary material:
    • The entire TensorFlow (or PyTorch, etc.) Github source code. Only put the code you have written for the project.
    • Any code that is larger than 10 MB.
    • Model checkpoints.

Submission: You will submit your final report as a PDF and your supplementary material as a separate PDF or ZIP file. We will provide detailed submission instructions as the deadline nears.

Additional Submission Requirements: We also ask you do the following when you submit your project report:

  • Your report PDF should list all authors who have contributed to your work; enough to warrant a co-authorship position. This includes people not enrolled in the course, such as faculty/advisors if they sponsored your work with funding or data, and significant mentors (including PhD students or postdocs who coded with you, participated in data collection, or helped draft your model on a whiteboard). All authors should be listed directly underneath the title on your PDF. Include a footnote on the first page indicating which authors are not enrolled in the course. All co-authors should have their institutional/organizational affiliation specified below the title, and their role should be described in the Contributions section of the paper.
  • Any code that was used as a base for projects must be referenced and cited in the body of the paper. This includes course assignment code, and fine-tuning example code, open-source, or Github implementations. You can use a footnote or full reference/bibliography entry.
  • If you are using this project for multiple classes, submit the other class PDF as well. Remember, it is an honor code violation to use the same final report PDF for multiple classes.

Grading Breakdown

(10%) Project Proposal
(20%) Project Milestone
(5%) Project Midpoint Presentation
(5%) Project Advising Session with TAs
(10%) Project Final Presentation
(50%) Project Report

Honor Code

You may consult any papers, books, online references, or publicly available implementations for ideas and code that you may want to incorporate into your strategy or algorithm, so long as you clearly cite your sources in your code and your writeup. However, under no circumstances may you look at another group’s code or incorporate their code into your project. If you are combining your course project with the project from another class, you must also follow all guidelines for this as specified above.

Option to Collaborate with BIODS 388

There is an option to collaborate with students in BIODS 388 (Stakeholder Competencies for AI in Healthcare, a non-programming course targeted for students from healthcare / clinical backgrounds) for the final project. BIODS 388 students have a final project that involves writing a paper in clinical journal format for an AI+healthcare application. You may choose to collaborate with BIODS 388 students such that they write their clinical paper focused on your BIODS 220 project. This could produce complementary insights, since a clinical journal format would focus more heavily on the clinical problem and background, compared to the more technical focus of BIODS220. Some of these students may also bring complementary expertise from medicine, health policy, business, etc. that could be useful for your project. Note that students from BIODS 388 are not expected to contribute to the engineering or programming component of BIODS 220 projects. The project group size used for grading will also count only BIODS 220 students; any BIODS 388 students would be considered as external collaborators (see “Additional Submission Requirements” in the Final Project Report section of this page). There will be a project partner finding session during week 1 that will allow you to get to know students in BIODS 220 as well as BIODS 388 for team formation.

Project Ideas

While you have significant flexibility with regards to the precise project topic, examples of types of projects you may wish to pursue include:

  • Novel problem
    • Develop a deep learning model for a new problem that has not previously been experimented on.
    • Utilize a new dataset or tackle new tasks on existing data
  • Novel analysis
    • Reimplement an existing paper that you find interesting.
    • Try different algorithmic variants to see if they improve performance.
    • Analyze and try different types of input data.
    • Interpret the model in different ways.
    • Predict different outputs than current state of the art.
  • Novel methods
    • New algorithms or model architectures for an existing problem.
    • Improve upon current problem’s measure of success.

Regardless of the nature of your project, grading will be based on the thoughtfulness, thoroughness, and depth with which you address the required components in each of the project deliverables.

Possible sources of project inspiration include published papers, Kaggle competitions, and public datasets and repositories including the following:
Biomedical images: Grand Challenges in Biomedical Image Analysis
Electronic health records: MIMIC Critical Care Database
Genomics: ENCODE (Encyclopedia of DNA Elements)

We will also be sharing a list of project ideas contributed by members of the Stanford community through Piazza.

Autumn 2020-2021 Projects

  • Coronavirus Classification Using Temporal Convolution Networks
  • Multimodal Semantic Embeddings to Reduce Hidden Stratification in Medical Imaging Data
  • Prediction of Gene Expression from Histopathology Images via Deep Learning in Gastric Cancer
  • Scaling Graph Neural Networks for Drug-Drug Interaction Prediction Using Partitioning and Parallelization
  • Skin Lesions: HAM 10000 dataset
  • Predicting Need for Medical Examination of Skin Lesion using Segmentation and Ensemble Models
  • Predicting Mental Illness in the Tech Industry
  • Image Classification for Short Time Fourier Transformed ECG Signals of Cardiac Arrhythmias
  • Multiclass Segmentation on Prostate Cancer (PCa)
  • Predicting Birth Outcomes for In-Vitro Fertilization using Embryonic Images
  • Deep Learning for PRedicting Hospital Admissions due to Chronic Disease Exacerbation during Periods of Bad Air Quality
  • Object Detection for Deep Veins of the Leg and Pelvis
  • Predicting Postoperative Cochlear Implant Performance using Multimodal Deep Learning
  • Investigating Self-Supervised Learning for Medical Imaging
  • Using Deep CNNs to Detect COVID-19 Pneumonia in Chest X-rays
  • Non-Contrast-to-Contrast CT Translation Generative Adversarial Networks for Rapid Detection of Ischemic Stroke
  • Prediction of DNA Methylation Level from DNA Sequence from Germ-line
  • Assessing Squat Form Using 3D Pose Data
  • Multi-modal Transformer Learning Medical Visual Representation – An Application Study in Detecting Underrepresented Cardiopulmonary Conditions
  • Using AI Algorithms for cryo-ET: Identifying Intracellular Macromolecules in Huntington’s Disease
  • Deep Learning Predicts The Occurrence of Persistent Post-Concussion Symptoms Following Mild-Traumatic Brain Injuries
  • Using Deep Learning to Detect Glaucoma from Different Visual Modalities
  • Predicting Malnourishment based on Electronic Health Records
  • Comparison of Embedding Strategies for LSTM based RNN model for Medical Prediction
  • Comparing CNN and LSTM Methods for Fall Detection
  • Drug-Target Interaction Prediction Using Multi-modal Features

ORDER THIS PAPER HERE

Study Selection and Summary

The Selected Classic Study from the Ten Classic Studies List

#5. Bystander Effect – Helping Behavior – John Latane and Bibb Darley

Part 1: Summary of the Selected Classic Study

Research question

The research question is, “What factors affects a bystander’s intervention in emergencies?”

Literature review

There are some shared human values regarding assisting the victim during problems, but there are also logical and illogical fears concerning what may befall the person who intervenes. In some situations, the value of necessitating intervention may be reduced, making bystanders address the situation differently. For instance, when there are other onlookers on the scene of emergencies, a bystander may feel that the responsibility for intervening is collective by not individual. In this case, every bystander thinks that the responsibility of helping is distributed or diffused, among other onlookers (Darley & Latané, 1968). 

Research methodology 

This experimental study involved 59 female and 13 male students who had been enrolled in introductory psychology programs at New York University. The participants took part in this experiment as partial fulfillment of their classwork. The participants were to give their opinions regarding the forms of issues facing college students in a populated urban setting. 

Research results 

It was found out that a person is usually reluctant to intervene in an emergency when he or she is aware that other onlookers are present. As such, the victim is more likely to get help when there is a single or few observers than when there are multitudes of onlookers/bystanders. 

Research conclusion

The study suggests that a bystander’s nonintervention in real-life emergencies lie more in his or her reaction to the other onlookers than his or her feeling about the victim. 

Part 2: Summary of a Study by Wilhelm & Bekkers (2010)

Research question

The research question is, “What is the relative significance of the innate human norm of helping others and the principle of care?”

Literature review

When individuals observe the needs of others, they react with empathy. In this case, a person’s empathy makes them help others. However, a person is not necessarily triggered to be empathic when observing the needs of others, but it is because they have an innate value of wanting to help. Dispositional empathic concern refers to a situation where an individual experiences concern, sympathy, and compassion when reacting to problems facing other people. The principle of care, on the other hand, relates to a person’s moral duty in helping the needy (Wilhelm & Bekkers, 2010). 

Research methodology 

The study used the data from the General Social Survey to examine behaviors that individuals have in helping others. The independent variables of this study are a dispositional empathic concern and the principle of care. On the other hand, the dependent variables of the study are different forms of helping behavior. Random sampling was used in selecting the participants of the study from the U.S. adult population.  

Research results 

The study found out that the care-helping relationship is greater than the empathy-helping relationship for many actions involving helping people. It was also established that empathy-helping association is founded on the human norm of assisting people. 

Research conclusions 

The study suggests that the principle of care reconciles innate norm of an individual in helping others. 

References

Darley, J. M., & Latané, B. (1968). Bystander intervention in emergencies: Diffusion of    responsibility. Journal of personality and social psychology, 8(4), 377–383.

            doi: 10.1037/h0025589

Wilhelm, M. O., & Bekkers, R. (2010). Helping behavior, dispositional empathic concern, and      the principle of care. Social Psychology Quarterly, 73(1), 11-32.

https://doi.org/10.1177/0190272510361435