Discuss: Image Classification using Foundation Models

[birm]

Discuss, or Question and answer about this project specifically.

@tkurc

[AnonymousCodes911]

Greetings @birm !
I'm intrigued by this project and would love to learn more about it. Any additional insights you could provide would be highly appreciated. Also, could you please offer any tips on how to get started with this?
Thank you!

[AanshSamyani]

Hi @birm
I came across this project and loved the idea. I have good knowledge about leveraging pre-trained foundation models or fine-tuning them for image classification. I would love to contribute to such a project. Can you guide me how I should start building a roadmap for this project?
Thank you!

[Satvik713]

Hey @birm
I am excited about the prospect of contributing to this project. I have been working on a few projects on topics including computer vision and large language models. I would like to use my knowledge about transformer models and implement a transformer infused image classification models for this task. I am enthusiastic about taking on contributor tasks to enhance this initiative.
Thank you

[katg999]

Hello @birm
Would be grateful to work on this idea

[Felicia-fang]

Hello @birm. As a Master’s student deeply engaged in finetuning pre-trained models for 3D object detection and skilled in 2D image classification, I am drawn to this project. May I discuss the project specifics with the mentors prior to application submission? Also, I wonder if there are any assessments to showcase my proficiency relevant to this project.

[birm]

We've added some basic "contribution tasks" as a starting point -- https://github.com/camicroscope/GSOC/blob/master/ContributorTasks.md

I can't speak to how Tahsin would look at applications, but the questions I tend to ask myself when looking at applications are "do they understand the problem?", "do they seem like they can do the core of the work", and "will they have substantial challenges getting the code working with caMicroscope". Things like contribution tasks I think help address the second two questions, but aren't the only ways to do so. I'll ask Tahsin about this too; update to follow soon hopefully!

[aliceheiman]

Hello @birm and everyone!

My name is Alice and I am a Computer Science undergrad at Stanford University pursuing the Computational Biology track.

I am very interested in joining this project as I am passionate about contributing to the intersection between AI and health.

• The course PSYC221: Machine Learning for Neuroimaging introduced me to the exciting possibilities of combining neuroscience and AI. I used Scikit-Learn and PyTorch to apply and build machine-learning models for brain analysis. For example, I recreated a U-net for MRI segmentation and built an encoder-decoder model to generate artificial brain images.
• The course CS173A: A Computational Tour of the Human Genome gave me skills in working with genomic data and the genome browser using Python, Bedtools, and Unix.

I have a strong foundation in Python and HTML/CSS, and also have experience in C, PHP, and JavaScript.

I would love to contribute to the project! Just one idea for further development:

• Do exploratory data analysis on images and take a "quiz" about the data to identify the most suitable architecture for the data.

I look forward to discussing this project further with you!

Warm regards,
Alice :)

[Nouran-Muhammad]

Greetings,
I am Nouran Muhammad, a biomedical engineer who graduated from Cairo University 2023. I am interested in applying AI in biomedical engineering projects. I read about caMicroscope and your projects for google summer of code.

I am interested in the project: Image Classification using Foundation Models and I have 2 questions regarding the project required skills:
1- UX: are the Frontend and software components limited to a specific framework or a programming language?
2- Micro services: can you explain in detail what is exactly meant by this requirement?
Thank you

[birm]

for 1, I think the core requirement is that it can work with caMicroscope in the web.
For 2, microservices are little self-contained sets of usually api endpoints. So for machine learning, you may have some special api routes (e.g. train model) that you need to package separately from caMicroscope's nodejs based backend.

[Nouran-Muhammad]

I understand that the machine learning module will need python experience which I have, and I understand the API endpoints part. But, my question is that caMicroscope is built using nodejs, HTML, CSS, and the project is a new tool to be added to the website, so does the project require knowledge with nodejs?
Thank you

[birm]

Since it's pytorch, It'd be an a different service, like ca-load for example.

[birm]

I think Tahsin is a bit busy (as are we all), so I'd like to ask that any other questions you'd like me to summarize to Tahsin and report on the results. I'm of course including everything currently here and answering the questions that I can!