Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Minor WebDev | Real-time Web
Dave Bitter
Mirror Stream
V 1.1.1
General
This repository holds the code for the course 'Real-time Web', part of the minor "Webdevelopment" at the HvA.
Mirror Stream let's user view their most used hashtags and most popular hashtags on the app. With these hashtags, streams of posts worldwide containing that hashtag can be opened.
Issues
I highly encourage adding issues to this repo in the case of finding one. I'm always up for improving my code.
Table Of Contents
Features
- Instagram oAuth authentication
- Show user's latest post and all the used hashtags
- Open stream with post with the hashtag a user selects through Socket.io
- Most clicked hashtags, only sent if new information (MongoDB)
- Service Worker and Progressive Web-app
- API Error Handling
Instagram oAuth authentication
The app is limited due to Instagram's strictness with their API. The app is currently is Sandbox mode.
TL:DR;
Firstly, I have to authorize your Instagram account as a Sandbox-user for this app.
Secondly, you have to accept this invitation here. Note that you have to have developer account at Instagram to view this page.
Great! You should be able to now go through the oAuth flow. Final thing though. The posts on Instagram that can be accessed in the app is limited to the posts of the Sandbox users. This means that the stream won't show new posts at a crazy rate, because their are a few users, the Sandbox users, that post.
This should al be fixed when the app is authorized by Instagram, which is a pain in the butt. Please accept this for now.
Show user's latest post and all the used hashtags
The user can see the last images they posted to Instagram, nicluding the hashtags per post. On a second column, the user can see a list of all the hashtag used in these posts together.
Open stream with post with the hashtag a user selects through Socket.io
A user can click on one of the hashtags to open up a stream with post that contain that hashtag. Every X amount of seconds a fresh set of posts is send over the socket connection. Which user gets which posts based on their hashtag is stored on the server side of this application.
Most clicked hashtags, only sent if new information (MongoDB)
Whenever a user clicks on a hashtag the count for that hashtag goes up. Every X amount of seconds the top ten hashtags get send to the 'Top Hashtags By Users' section. Each time, the server will check whether the information is identical to the previously send information. If this is true, nothing will be sent. The storing of this data happens with MongoDB.
Service Worker and Progressive Web-app
If connection between the client and the server is lost the Service Worker kicks in. It will display a user with feedback about what possibly happened. It will also link to Instagram's hashtag search page so the user can still look at the posts for that hashtag.
API Error Handling
If the API throws an error the error will be handled by showing the user that something went wrong along with the statuscode.
If the api goes down and we can do a request a page will be displayed in order with a link to the instagram search page.
Emitted Socket Events
Server
connected users (on event)
This will emit the current amount of users that are online on the app on the event of a user that connects or disconnect.
top tags (on event)
This will emit the top clicked hashtags by users every X amount of seconds. The information will be queried on the MongoDB and only emitted if the data is different than the previous emit.
new tagstream (stream)
This will emit the latest posts for the selected hashtag per user every X amount of seconds.
no tagstream (on event)
This will emit the statuscod. On the client the message will be displayed at the top of the feed.
Client
connection (on event)
This will emit the new connection with the socket information (like the socket id) to be registered on the server side.
new tag (on event)
This will emit a newly selected hashtag by the user on the client side to the server side. On the server the client's hashtag to stream posts with will be updated aswell as the 'Top Hashtags By Users'.
Tooling
This app doesn't run on much packages.
Express (session)
Used for setting up routes and storing user data in session.
MongoDB
Used for storing most clicked hashtags of all clients ever connected.
Pug
Used for rendering front-end. I decided to use Pug beacuse of it's uncluttered syntax.
Request
Used for API calls to the Instagram server.
Socket.io
Used to setup and handle all socket connections from clients to own server.
Wishlist
- Data visualization of events like most clicked hashtags, new clients online etc
- Less clientside templating
- More interaction with posts like follow the person who posted, liking, commenting etc.
Live Demo
Taken al this in account, you can view the app here.
Getting started
Clone this repo, duh
git clone https://github.com/DaveBitter/minor-webdev_real-time-web.git
cd minor-webdev_real-time-web
Install the dependencies
npm i
Populate the .env file
Save the example.env as env and fill in the values.
You will need to get the keys from the Instagram API aswell as set-up a MongoDB database.
Start up the server
When you run this command, changes in server side JS files will be watched and the server will restart automatically, changes in client side JS files will be watched and browserified and the server will be restarted.
npm start
Additional commands
Browserify client side JS
npm run build
Watchify client side JS
npm run watch
Clean client side build.js file
npm run clean