GePSAn: Generative Procedure Step Anticipation in Cooking Videos
Mohamed A. Abdelsalam, Samrudhdhi B. Rangrej, Isma Hadji, Nikita Dvornik, Konstantinos G. Derpanis, Afsaneh Fazly.
ICCV, 2023
We've developed a generative deep learning model to predict the next steps in procedural videos using natural language. This model, unique in its ability to generate diverse and plausible outcomes, was pretrained on a large text corpus before application to video data. It sets new benchmarks on the YouCookII dataset and is capable of zero-shot transfer from text to video, showcasing versatility and accuracy in predicting future steps without any need for fine-tuning.
Mohamed Ashraf Abdelsalam
I work as a Machine Learning Research Engineer at Samsung AI Center (SAIC) Toronto. Before that, I did my Master’s in Machine Learning at Mila and the University of Montreal, where I was supervised by Dr. Sarath Chandar . In 2018, I completed my Bachelor’s in Aerospace Engineering from Zewail University of Science and Technology. Check out my personal CV for more details!
Research
Broadly speaking, my research interests lie in the field of machine learning and deep learning. Currently, I am primarily focused on multimodal research research in the intersection of vision and language. Additionally, I am also interested in the area of continual learning and the development of flexible models that can accumulate new knowledge, modify existing knowledge, and learn new tasks without forgetting previously acquired knowledge and experiences, even in the absence of data or with limited data available for these prior tasks.
Publications
Visual Semantic Parsing: From Images to Abstract Meaning Representation
Mohamed A. Abdelsalam, Zhan Shi, Federico Fancellu, Kalliopi Basioti, Dhaivat J. Bhatt, Vladimir Pavlovic, Afsaneh Fazly.
CoNLL, 2022
The paper proposes a new method for visual scene understanding by using Abstract Meaning Representation (AMR) to create linguistically informed visual AMR graphs that focus on higher-level semantic concepts, and demonstrate the feasibility of the approach through experimentation and analysis.
Mohamed A. Abdelsalam, Zhan Shi, Federico Fancellu, Kalliopi Basioti, Dhaivat J. Bhatt, Vladimir Pavlovic, Afsaneh Fazly.
CoNLL, 2022
The paper proposes a new method for visual scene understanding by using Abstract Meaning Representation (AMR) to create linguistically informed visual AMR graphs that focus on higher-level semantic concepts, and demonstrate the feasibility of the approach through experimentation and analysis.
A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss
Prasanna Parthasarathi, Mohamed A. Abdelsalam, Joelle Pineau, Sarath Chandar</i>
SIGDial, 2021
The study explores the use of an alternate training objective, minimizing a semantic loss as an auxiliary objective, to foster generating alternate responses and improve diversity in next utterance generation in dialogue tasks, with observations showing improvement in diversity in smaller datasets.
Prasanna Parthasarathi, Mohamed A. Abdelsalam, Joelle Pineau, Sarath Chandar</i>
SIGDial, 2021
The study explores the use of an alternate training objective, minimizing a semantic loss as an auxiliary objective, to foster generating alternate responses and improve diversity in next utterance generation in dialogue tasks, with observations showing improvement in diversity in smaller datasets.
IIRC: Incremental Implicitly Refined Classification
Mohamed A. Abdelsalam, Mojtaba Faramarzi, Shagun Sodhani, Sarath Chandar.
CVPR, 2021
A setup and benchmark for evaluating lifelong learning models in more dynamic and realistic scenarios.
Homepage | arXiv | Documentation | Code
Mohamed A. Abdelsalam, Mojtaba Faramarzi, Shagun Sodhani, Sarath Chandar.
CVPR, 2021
A setup and benchmark for evaluating lifelong learning models in more dynamic and realistic scenarios.
Homepage | arXiv | Documentation | Code
Primers
An Introduction to Lifelong Supervised Learning
Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta, Pranshu Malviya, Mohamed A. Abdelsalam, Janarthanan Janarthanan, Sarath Chandar.
This primer is an attempt to provide a detailed summary of the different aspects of lifelong supervised learning.
Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta, Pranshu Malviya, Mohamed A. Abdelsalam, Janarthanan Janarthanan, Sarath Chandar.
This primer is an attempt to provide a detailed summary of the different aspects of lifelong supervised learning.