Improved Intra Server v3.3.0

New features

You can now synchronize events and exams you have registered to with your personal calendar app (if it supports the iCal protocol)!

Setting it up is quite easy. Head over to the Calendar Synchronization page in the extension's settings, then click the blue button that says "Add the Intra calendar to your calendar app". Alternatively, you can copy the URL by right-clicking the button to import the calendar manually.

Events and exams you have registered to are synchronized every 3 hours. When unregistering from an event, it will not get deleted from your calendar.

Other changes

• Increased security in the Extension Token authentication system; newly generated tokens for the same user are now always slightly different
• Hid the option to display progress towards Codam's Monitoring System, as this feature is no longer supported by the browser extension
• Implemented GitHub Action to test automated dependency updates using Dependabot
• Fixed localhost IP address in the takeout ZIP's readme (takeout = "request a copy of my data" in the Help section)

Breaking changes

Before pulling this release, the database needs to be modified manually:

ALTER TABLE runners ADD COLUMN events TIMESTAMP WITHOUT TIME ZONE;

Improved Intra Server v3.2.1

This hotfix update includes three major bugfixes:

• Prevent gunicorn workers from crashing on failed user authentication with the Intranet (fixes FreekBes/improved_intra#57)
• Updated dependencies (dropped support for OpenSSL 1.1.1d and older)
• Temporary workaround to prevent spamming of the Intra API by runners (specifically the outstandings-data-fetcher). This workaround degrades performance: it reduces the amount of gunicorn workers from 4 to 1. A permanent fix will be released in the future, by separating the runners from the workers.

Not updating to this release could potentially cause the IP of the server to get blacklisted by 42's fail2ban system because of the API spam from the 4 workers, preventing the server from working correctly.

Because of the 4 workers each fetching outstandings individually, outstanding marks might not be displayed correctly on user profiles. To fix this issue, the database tables containing these marks need to be cleared. Apply the following SQL commands to the postgres database:

START TRANSACTION;
DELETE FROM teams;
DELETE FROM evaluations;
UPDATE runners SET outstandings=NULL;
COMMIT;

The database tables will get fully repopulated on the next run of the outstandings-data-fetcher runner.

Improved Intra Server v3.2.0

Changelog

• Added option to sort completed projects on user profiles by date instead of alphabetical order (requires Improved Intra v4.1.0 or higher)

Breaking changes

Before pulling this release, the database needs to be modified manually:

ALTER TABLE settings ADD COLUMN sort_projects_date BOOLEAN;
ALTER TABLE settings ALTER COLUMN sort_projects_date SET DEFAULT 'f';
START TRANSACTION;
UPDATE settings SET sort_projects_date='f';
COMMIT;
ALTER TABLE settings ALTER COLUMN sort_projects_date SET NOT NULL;

Improved Intra Server v3.1.1

Bugfixes

• Fixed session management bugs (including possible fix for issue FreekBes/improved_intra#52)
• Reworked authentication decorators

Improved Intra Server v3.1.0

New privacy-related features

This update of the Improved Intra back-end server focuses on privacy-oriented features.

Requesting a copy of your personal data

An option to request all data Improved Intra stores about you, as an end user, has been added to the settings. You can find this option under the Help section.

When clicking on the link to request all your data, a ZIP file will be generated by the back-end that contains all data it holds on you. The archive contains multiple JSON files, each representing a part of the data used by the back-end. Here is an overview of the contents:

• banners: this folder contains all banners you have previously uploaded to the back-end server, that are still on the server. Normally, any banner that has been uploaded more than a month ago and is no longer in use will be deleted.
• campus.json: contains information about the campus you're attending. It's not much, but it's related to you.
• oauth_tokens.json: contains information about your Intra OAuth2 tokens. For security reasons, the tokens themselves are not included.
• profile.json: contains all custom Improved Intra features you choose to display on your Intra profile, such as the links to your Git, website and the currently selected banner image.
• project_users.json: contains the Intra teams for all projects you've done for all cursuses during your studies at 42, plus the marks for each of their evaluations. We use this data to calculate the amount of outstandings for each of your projects. Nothing more than needed is stored.
• readme.txt: contains a piece of text marking when this archive was created.
• settings.json: contains all of the general settings you're using for the Improved Intra browser extension.
• user_tokens.json: contains information on your Improved Intra sessions. The token used to authenticate you from the front-end of the extension, is of course not included.
• user.json: contains some personal information related to you, such as your name(s), login, whether you are staff, your e-mail address and when the Intranet says they will anonymize your account.

You can only request your own data, not the data of other Improved Intra users. And, as a general reminder: we do not store any data of any students who have not used Improved Intra in the past.

Account anonymization / deletion of your data

Account anonymization / deletion of your data has not been implemented yet. This is planned for a later update. In the meantime, you can follow the procedure listed in the Help section under "Privacy", if you wish to delete your data. Freek will then do this manually.

Bugfixes

• Fixed JSON parsing error when uploading banners that exceed 10MB in file size
• Fixed selected file not being cleared when clicking the "Unset" button to remove the banner from your Intra profile
• Fixed duplicate teams appearing in the output of the v1 outstandings endpoint
• Reduced API calls after OAuth2 authentication with the Intranet
• The "Last used at" column in the User Tokens table is now used. This value will be set to the timestamp of the last authentication done using the user's authentication token. This can be useful for tracking which user sessions have been recently active.
• Fixed potential security issues in dependencies by updating them: cryptography (PR #4) & werkzeug (PR #5)

Improved Intra Server v3.0.2

Improvements

• Significantly increased performance on the outstandings endpoint (from sometimes >12 seconds to <250 milliseconds)
• Added runner to delete unused banners that are older than one month. This runner runs every night and decreases storage usage.
• Moved logs to logs folder
• Bumped various dependencies

Breaking changes

Because the logs are moved to the logs folder, make sure to create this directory after updating and before restarting the server: mkdir logs && chown www-data:www-data logs. Otherwise the server will refuse to start.

Improved Intra Server v3.0.1

Bugfixes

• Fixed a bug that prevented custom banner images from being uploaded
• Fixed a bug that caused an Internal Server Error upon loading the "Campus-specific settings" page for campuses without specific settings
• Fixed an Internal Server Error that would occur when the request to fetch the user's Intra image had failed and the session contains no user image
• Fixed an issue that prevented extension tokens from being generated for accounts without user authentication tokens (UserToken table)
• Fixed Internal Server Errors without proper error formatting on the /v2/ext_token endpoint

Improved Intra Server v3.0.0

New features

• Added a homepage at https://iintra.freekb.es (more will be added here in the future)
• Revamped settings page for Improved Intra v4.0.0
• New authentication system between the extension and the back-end server: no longer are access tokens for 42's Intra API shared with the frontend

Requirements

This version requires several more additional pip packages. After git pull, make sure to run pip3 install -r requirements.txt again (in the virtual environment) - only then restart the server with systemctl restart iintra-server.service.

Improved Intra v4.0.0

This version is now targeting v4.0.0 of the extension. While it still supports version 3.5.0 and older, support for versions 3 and below will be dropped in the future. More about this change can be found in the release notes for Improved Intra v4.0.0.

Improved Intra Server v2.0.1

A small update of the Improved Intra back-end this time.

Features

• Added v1's imagery.php page back, but now for campus staff only
• Add SQL caching

Bugfixes

• Fixed outstandings not displaying for team projects on some user profiles
• Update requirements

Improved Intra Server v2.0.0

The first release for the back-end of Improved Intra!

Why a release schedule for the back-end?

The reason for this release (and future ones) is that in the future, campuses will be able to create a modified version of the back-end server (and front-end extension) if they wish to. Or specific groups of users, any possibility is out there. I am planning on creating features that allow those servers to communicate with the main back-end at https://iintra.freekb.es/, while allowing them to provide additional features that I do not offer. I do not think this feature will be used much or by many, but it could be nice nonetheless. I will certainly use it myself for debugging purposes (no more working in production...)!

Another reason for a release schedule for the back-end is to provide more open-ness about how student data is managed in the back-end. As the extension is nearing 1500 users, and I joined Codam's staff team, I wish to be more transparent about Improved Intra. Nothing to worry though, it's only getting better! Read on to know how...

More privacy

Starting from v2 of the back-end, your data will be more securely managed. Instead of every setting being saved in a public JSON file per user, I am now using a postgresql database, and the endpoints will in the future no longer share all settings/data with everyone. Yes, some session management will have to be implemented, but that is for another future release. As the current and old versions of the extension are still expecting all data to be public, I have written the new back-end so that the old *.php endpoints still return all data expected by those versions. The v2 endpoints will feature more strict privacy, which will also allow me to bring some more user-customized features to the extension (and to Intra)!

Full changelog

• Switched from PHP to Python
• Now using Flask in combination with gunicorn and nginx on Debian
• Switched from settings being stored in JSON files in folders to an actual database (it was about time, really)
• Re-implemented the OAuth2 system
• Better form validation
• Implemented Hive Helsinki's 42 API wrapper
• New endpoints:
  • /auth: authenticate users using their 42 Intra account and start a session
  • /v2/connect: authenticate users for the future v2 back-end api (redirects to new /auth endpoint)
  • /v2/disconnect: log out from a session
  • /settings/.json: for fetching a user's settings (for extension versions < 3.0.0)
• Recreated the following routes:
  • /options.php: for modifying extension settings for extension versions < 3.0.0 (version 3.0.0 to be released later this year)
  • /connect.php: for authenticating using Intra accounts (redirects to new /auth endpoint, then /options.php)
  • /update.php: for storing user settings
  • /outstanding.php: for fetching outstandings of users
  • /testkey.php: for testing if a session is still active for extension versions < 3.0.0
• Removed the following routes:
  • /delete.php: for deleting a user's settings (was never accessible anyways, this feature will be brought back in v3.0.0 of the extension)
  • /buildinghours.php: for retrieving a Codam student's building hours (was never fully implemented)
  • /imagery.php: for viewing uploaded banners

Codam

With the new monitoring system coming soon, I will re-implement the Monitoring System progress feature. I have plans for it, but am yet to develop them and am still unsure of how to implement those plans fully. Something is on the horizon though!