Designing INSTAGRAM: System Design of News Feed

About this video

### Summary of Designing Instagram - System Design 1. **Introduction and Key Features**: - The video discusses designing Instagram, focusing on four key features: 1. Storing and retrieving images. 2. Liking and commenting on posts. 3. Following other users. 4. Publishing a newsfeed. 2. **Storing and Retrieving Images**: - Similar to Tinder's design, use a file system with a CDN for fast access. - Emphasizes the need for cheaper storage solutions. 3. **Liking and Commenting on Posts**: - **ER Diagram**: - `Like` table includes: post ID, user ID, timestamp, and active status. - Comments can be liked but not nested (no replies to replies). - Parent ID in the `Like` table indicates whether it refers to a post or comment. - **Optimization**: - Use an activity table to track likes and comments efficiently. - Avoid storing aggregated data (like counts) in the main tables; instead, compute them when needed. 4. **Following Users**: - Simple table structure: - Follower user ID. - Followee user ID. - Timestamp of when the follow action occurred. - Queries: - Find followers of a user. - Find users followed by a user. 5. **Publishing a Newsfeed**: - **Initial Approach**: - Retrieve posts from followed users each time a feed is requested. - Inefficient due to multiple queries and high load. - **Optimized Approach**: - Pre-compute user feeds: - Update feeds incrementally when new posts are made by followed users. - Store pre-computed feeds in cache (e.g., Redis) for quick access. - Limit the number of posts stored per user (e.g., top 20 posts). 6. **System Architecture**: - **Gateway**: - Handles external protocols and security mechanisms. - Encapsulates reverse proxy functionalities. - **Load Balancer**: - Distributes requests using consistent hashing based on user IDs. - Maintains snapshots of server states to route requests efficiently. - **User Feed Service**: - Depends on Post Service and Follow Service. - Optimized using pre-computed feeds and incremental updates. - **Other Services**: - Image Service, Activity Service, Chat Module, Profile Service, etc. 7. **Handling Celebrity Posts**: - Celebrities with millions of followers require special handling: - **Push Model**: Risky due to high fan-out rates. - **Pull Model**: Followers periodically pull updates, reducing server load. - **Hybrid Model**: Use push for normal users and pull for celebrities. 8. **Notifications**: - Notifications can be pushed via web sockets or pulled by clients. - Efficient routing of notifications to multiple followers discussed in subsequent videos. 9. **Conclusion**: - This architecture efficiently handles the core features of Instagram. - Understanding trade-offs between push and pull models is crucial. - Interviewers appreciate direct answers and advanced topic discussions. - Further details on specific services like chat modules will be covered in future videos. 10. **Additional Tips**: - Keep ER diagrams flexible and consider future requirements. - Simplify tables if necessary but ensure scalability. - Be prepared to discuss additional features like messaging, profile management, etc., during interviews. By following these structured steps and considerations, one can effectively design a scalable and efficient system akin to Instagram.

Course: System Design Playlist

**Course Description: System Design Playlist** This comprehensive course, titled "System Design Playlist," is designed to provide students with a deep understanding of system design principles and practices through real-world analogies and technical explanations. The course begins by using the analogy of running a pizza restaurant to illustrate fundamental concepts in system design, such as optimizing processes, scaling resources, and ensuring resilience. Students will learn about vertical scaling—enhancing the capabilities of existing resources—and horizontal scaling—adding more resources to distribute the workload. Through this engaging example, participants will grasp essential strategies for improving throughput, eliminating single points of failure, and implementing backup systems to maintain operational continuity. As the course progresses, students will delve into advanced topics like microservice architecture, where responsibilities within a system are clearly defined and divided among specialized teams or services. This approach allows for efficient scaling and management of different components based on their specific needs. Additionally, the course covers distributed systems, highlighting the importance of fault tolerance and quick response times by strategically placing servers closer to users. Concepts such as load balancing, which intelligently routes requests to optimize performance, and decoupling systems to enhance flexibility and adaptability, are thoroughly explored. Participants will also learn about logging and metrics to monitor system health and make informed decisions. The course wraps up by contrasting high-level system design, which focuses on overarching architectural decisions, with low-level system design, which deals with the actual coding and implementation details. By mapping business scenarios to technical solutions, students will gain insights into designing scalable, reliable, and extensible systems. Whether you're new to system design or looking to deepen your expertise, this course equips you with the knowledge and tools needed to tackle complex design challenges and develop robust systems capable of meeting diverse user demands.