Search and folder contextualized content recommendations

[rtibbles]

Background

We've worked to develop deep learning models for providing content recommendations. These models use contextual information, such as the title, description, and metadata of a topic/folder content node and its ancestors, to provide content recommendations from our public catalog of channels. Eventually, the end goal is to use these capabilities for curriculum alignment by taking a curriculum skeleton (parsed from curriculum documents) and filling in the structure with content using this recommendation engine.

Story

As a user opted-in to new 'AI' Studio capabilities, who is interested in curating the most applicable resources from Studio's public catalog for my channel, I would like the ability to obtain a list of recommendations for a channel folder I'm editing. I would like the ability to then import selected resources from those recommendations which I find applicable to my channel and its folder.

Requirements

• All UI features should be controlled by the 'Al' capabilities feature flag
• Deep learning models should be hosted on HuggingFace
• The backend should properly restrict outbound API calls to HuggingFace by validating the user has the feature flag enabled

Out of Scope

• Building the feedback architecture to track implicit and explicit feedback on recommendations, except for integrating that architecture into search recommendations

In-progress/Complete Tasks

Preview Give feedback

1. Create a new Django app to house all AI related functionality in Studio #4280
   DEV: backend P1 - important TAG: new feature +3
   
2. Development of Flexible API Layer #4275
   DEV: backend P1 - important TAG: new feature +3
   
3. Add a global getter that references the AI Flag in studio #4276
   DEV: frontend P1 - important TAG: new feature +3
   
4. Implement the Recommendations backend [boilerplate] #4296
   DEV: backend P1 - important TAG: new feature +3
   
5. Implement the RecommendationsAdapter [Boilerplate] #4302
   DEV: backend P1 - important TAG: new feature +3
   
6. Implement the Whisper backend #4284
   DEV: backend P2 - normal TAG: new feature +3
   
7. Implement the CloudStorage backend #4285
   DEV: backend P2 - normal TAG: new feature +3
   
8. Search Recommendations Progress Checklist #4294
   DEV: backend P1 - important TAG: new feature TAG: research work-in-progress +5
9. Implement a manager class that interacts with the appnexus library #4295
   DEV: backend P1 - important TAG: new feature +3
   
10. Implement schema for embed API requests #4392
    DEV: backend P1 - important +2
    
11. Implement pre-processing for the embed api #4399
    DEV: backend P1 - important +2
    
12. Create a workflow that publishes docker container images to the GH container registry #4341
    DEV: backend DEV: dev-ops P1 - important TAG: new feature +4
    
13. Create a model that caches generated embeddings #4290
    DEV: backend P1 - important TAG: new feature +3
    
14. Implement the RecommendedResourceCard component in Studio to display recommended resources #4450
    DEV: frontend P1 - important +2
    
15. Implement the core functionality for the connect and make_request methods of the Backend abstract class. #4447
    DEV: backend P0 - critical +2
    
16. Implement logic to embed topics and content #4448
    DEV: backend P1 - important +2
    
17. Update the embed_content in RecommendationsAdapter #4455
    DEV: backend P1 - important TAG: new feature +3
    
18. Update Import from other channels page to display recommendations #4565
    DEV: frontend P1 - important TAG: new feature TAG: ux update +4
    
19. Implement About recommendation modal #4566
    DEV: frontend P1 - important TAG: new feature TAG: ux update +4
    
20. Implement pagination for recommended resources #4567
    DEV: frontend P1 - important TAG: new feature TAG: ux update +4
    
21. Implement a translator object and add strings for the search recommendations feature #4683
    DEV: frontend TAG: user strings +2
    

Feature Tasks

Preview Give feedback

1. Implement the feedback architecture
2. Implement exportation of content metadata and text in a format that recommendations models can understand
3. Setup the recommendations model infrastructure for production deploys
4. Improve developer experience while setting up studio for curriculum alignment development
5. Implement text extraction logic for content nodes of kind 'slideshow'
6. Implement an API endpoint to return recommendations
7. Implement logic that filters the recommendations based on the selected channel

To Spec / In Speccing

Preview Give feedback

No tasks being tracked yet.

Tasks

Preview Give feedback

We're having trouble rendering your tasklist, it's most likely has one of the following formatting errors:

• indented tasks (i.e. nested tasklist; support for this is coming!)
• an empty task (i.e. - [ ] on a line by itself)
• any empty new lines (before/after the tasklist or in between tasks)
• duplicate issue/pr links
• a draft task exceeds 512 characters

Please check the tasklists documentation for more information. 🔗 Feedback Discussion

Thank you for participating in the Private Beta ❤️

No tasks being tracked yet.

[vkWeb]

Looks promising, I'll be happy to help with this! ❤️