COMP5329 - Deep Learning
Assignment- 2
Assignment-2 has two tracks: competition track and reseach track.
Students should attend one of the two tracks. 2 or 3 students are suggested to form a group to attend one of these
two tracks.
1. Competition track description [ 100 Marks]:
In this assignment, you are to solve the multi-label classification task. Each sample in this dataset includes:
- an image,
- one or more labels,
- a short caption that summarizes the image.
Your goal is to implement an image classifier that predicts the labels of image data sample. You
may optionally include the caption in the input of your classifier – it’s up to you!
Please submit your submission file via Kaggle
https://www.kaggle.com/competitions/multi-label-classification-competition- 2023
Remember: the ranking contributes to 20% of your assignment mark.
Please make sure you name your team in the following format.
{unikey1}{unikey2}{unikey3}
The evaluation metric for this assignment is Mean F1-Score. The F1 score, commonly used in information
retrieval, measures accuracy using the statistics precision p and recall r. Precision is the ratio of true positives (tp)
to all predicted positives (tp + fp). Recall is the ratio of true positives to all actual positives (tp + fn). The F1 score
is given by:
The F1 metric weights recall and precision equally, and a good retrieval algorithm will maximize both precision
and recall simultaneously. Thus, moderately good performance on both will be favored over extremely good
performance on one and poor performance on the other.
Submission Format
For every image in the dataset, submission files should contain two columns: image id and labels. Labels should
be a space-delimited list.
For example
ImageID,Labels
1.jpg, 1
8.jpg, 8
9.jpg, 9 10
10.jpg, 10 9
etc.
You can use any methods in deep learning to accomplish the classification task. You must guarantee that the
submitted codes are self-complete, and can be successfully run in common python3 and PyTorch environment.
Instructions to hand in the assignment
- Go to Canvas and upload the report. The report should include each member’s details (student ID and name).
1 | If you work as a group, only one student needs to submit the report which must be named as student ID numbers |
- The report must include a link of your code, model and data (e.g., a shared Google cloud folder, so we can
easily run it on Colab). Clearly provide instructions on how to run your code in the appendix of the report or
include a readme.txt in your shared folder.
1 | In your shared folder, please include a file “Predicted_labels.txt”, which contains the predicted labels of test |
1 | Don’t update the code/data any more after the submission. If the latest modified time of the shared folder is |
- The report must clearly show (i) details of your modules, (ii) the predicted results from your classifier on test
examples, (iii) run-time, and (iv) hardware and software specifications of the computer that you used for
performance evaluations. - There is no special format to follow for the report but please make it as clear as possible and similar to a
research paper. - The use of ChatGPT or other AI tools is prohibited in the assignments. A plagiarism checker will be used.
Late submission:
Suppose you hand in work after the deadline:
If you have not been granted special consideration or arrangements
- A penalty of 5% of the maximum marks will be taken per day (or part) late. After ten days, you will be
awarded a mark of zero. - e.g. If an assignment is worth 40% of the final mark and you are one hour late submitting, then the maximum
marks possible would be 38%. - e.g. If an assignment is worth 40% of the final mark and you are 28 hours late submitting, then the maximum
marks possible marks would be 36%. - Warning: submission sites get very slow near deadlines.
- Submit early; you can resubmit if there is time before the deadline.
Marking scheme
1 | Category Criterion |
1 | Report |
1 | Introduction |
- What’s the aim of the study?
- Why is the study important?
- The general introduction of your used method in the assignment and your
motivation for such a solution.
1 | Related works |
- Existing related methods in the literature.
1 | Techniques |
- The principle of your method used in this assignment.
- Justify the reasonability of the method.
- Any advantage or novelty of the proposed method.
1 | Experiments and results |
- Accuracy/efficiency (Figures or Tables)
- Extensive analysis (ablation studies, comparison methods, hyper parameter
analysis)
1 | Conclusions and Discussion [5] |
- Meaningful conclusion and discussion.
1 | Other |
- At the discretion of the marker: for impressing the marker, excelling expectation,
etc. Examples include fast code, using LATEX, etc.
1 | Code |
1 | Code runs within a feasible time [5] |
1 | Size of resulting deep models for the prediction < 100MB [5] |
1 | Classification |
1 | Groups in top 10% [20] |
Penalties [-]
1 | Badly written : [-20] |
1 | Not including instructions on how to run your code: [-30] |
1 | Late submission |
2. Research track description [100 Marks]:
This research track calls for brave new ideas on deep learning. In this track, you are encouraged to propose and
investigate new algorithms or problems in deep learning.
You must not use the project (e.g. your honours, capstone or SSP project) that you have aleady done or are
currently doing in other units to participate in this track. You are encouraged to contact our teaching team, if you
want to have some discussion on your proposed research problems.
Submit the report (including a link to your source codes, models and data) on Canvas before the due date. TA
will further contact you for the preseantion slides by Week 13.
Marking scheme
1 | Category Criterion |
1 | Report |
1 | Introduction |
- What’s the aim of the study?
- Why is the study important?
- The general introduction of your used method in the assignment and your
motivation for such a solution.
1 | Related works |
- Existing related methods in the literature.
1 | Techniques |
- The principle of your method used in this assignment.
- Justify the reasonability of the method.
- Any advantage or novelty of the proposed method.
1 | Experiments and results |
- Accuracy/efficiency (Figures or Tables)
- Extensive analysis (ablation studies, comparison methods, hyper parameter
analysis)
1 | Conclusions and Discussion [5] |
- Meaningful conclusion and discussion.
1 | Other |
- At the discretion of the marker: for impressing the marker, excelling
expectation, etc. Examples include fast code, using LATEX, etc.
1 | Presentation [10] Presentation in week 13^ |
1 | Penalties [-] Late submission |