-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
# SIMS TO SFAS - INTEGRATION IMPLEMENTATION FOR STUDENT DATA RECORDS ## ENV - [x] Added a new env variable `SFAS_SEND_FOLDER`. Declared it as github env, not as secret. ## PROCESS - [x] File generated with following name structure `SIMS-TO-SFAS-YYYYMMDD-HHMMSS.TXT`  - [x] Student data is retrieved to build the file for all students who has one or more of the following updates since last run date. - Student or User data - Sin validation data - Cas supplier data - Overawards data - [x] Students with at least one submitted application are considered to extract the data. As submitted applications can be cancelled and also the Draft applications can be cancelled, application status NOT Draft could be tricky. So used the condition of application with current assessment NOT NULL to identify a submitted application. - [x] When one or more students with updates are present, then a file is produced and sent to SFAS SFTP location. **Summary:**  **Process Logs:**  - [x] When no students with updates are present, then no file is produced. **Summary:**  **Process Logs:**  ## TECHNICAL CONTEXT - STEPS involved in creation of the bridge data file ``` STEP 1 [Implemented]: Get the reference date of last bridge file sent. If there is no bridge file sent, it is null. STEP 2 [Implemented]: Get all the student ids of students who has one or more student data related change since last bridge file date. STEP 3 [TODO]: Get all the student ids and application details of students who has one or more application data related change since last bridge file date. STEP 4 [TODO]: Get all the student ids and restriction details of students who has one or more restriction data related change since last bridge file date. STEP 5 [Partially Implemented]: Consolidate all the studentIds produced in STEPS 2,3 and 4. STEP 6 [Partially Implemented]: Get student details of students who has one or more student data related change using consolidated studentIds. STEP 7 [Implemented]: Build header, footer and student records for file lines. STEP 8 [TODO]: Build application records and restriction records for file lines. STEP 9[Implemented]: Upload the file to SFTP and create bridge file log. ``` - For future implementations and areas, search for `TODO: SIMS to SFAS` in the code.
- Loading branch information
1 parent
4a2ac40
commit 7eedbb5
Showing
30 changed files
with
803 additions
and
15 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,3 +34,8 @@ export enum FullTimeAwardTypes { | |
| BGPD = "BGPD", | ||
| SBSD = "SBSD", | ||
| } | ||
|
|
||
| export enum YNFlag { | ||
| Y = "Y", | ||
| N = "N", | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 changes: 6 additions & 0 deletions
6
sources/packages/backend/libs/integrations/src/services/sfas/sims-to-sfas.model.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| import { Student } from "@sims/sims-db"; | ||
|
|
||
| export type StudentDetail = Student & { | ||
| cslfOverawardTotal?: string; | ||
| bcslOverawardTotal?: string; | ||
| }; |
177 changes: 177 additions & 0 deletions
177
sources/packages/backend/libs/integrations/src/services/sfas/sims-to-sfas.service.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,177 @@ | ||
| import { Injectable } from "@nestjs/common"; | ||
| import { InjectRepository } from "@nestjs/typeorm"; | ||
| import { StudentDetail } from "./sims-to-sfas.model"; | ||
| import { | ||
| BC_STUDENT_LOAN_AWARD_CODE, | ||
| CANADA_STUDENT_LOAN_FULL_TIME_AWARD_CODE, | ||
| } from "@sims/services/constants"; | ||
| import { | ||
| Application, | ||
| ApplicationStatus, | ||
| mapFromRawAndEntities, | ||
| SFASBridgeLog, | ||
| Student, | ||
| } from "@sims/sims-db"; | ||
| import { Brackets, Repository } from "typeorm"; | ||
|
|
||
| /** | ||
| * SIMS to SFAS services. | ||
| */ | ||
| @Injectable() | ||
| export class SIMSToSFASService { | ||
| constructor( | ||
| @InjectRepository(Student) | ||
| private readonly studentRepo: Repository<Student>, | ||
| @InjectRepository(Application) | ||
| private readonly applicationRepo: Repository<Application>, | ||
| @InjectRepository(SFASBridgeLog) | ||
| private readonly sfasBridgeLogRepo: Repository<SFASBridgeLog>, | ||
| ) {} | ||
|
|
||
| /** | ||
| * Get the latest bridge file log date. | ||
| * When there is no log, null will be returned. | ||
| * @returns latest bridge file log date. | ||
| */ | ||
| async getLatestBridgeFileLogDate(): Promise<Date | null> { | ||
| const [latestBridgeFileLog] = await this.sfasBridgeLogRepo.find({ | ||
| select: { id: true, referenceDate: true }, | ||
| order: { referenceDate: "DESC" }, | ||
| take: 1, | ||
| }); | ||
| return latestBridgeFileLog ? latestBridgeFileLog.referenceDate : null; | ||
| } | ||
|
|
||
| /** | ||
| * Log the details of bridge file that was sent to SFAS. | ||
| * @param referenceDate date when the bridge file data was extracted. | ||
| * @param fileName bridge file name. | ||
| */ | ||
| async logBridgeFileDetails( | ||
| referenceDate: Date, | ||
| fileName: string, | ||
| ): Promise<void> { | ||
| await this.sfasBridgeLogRepo.insert({ | ||
| referenceDate, | ||
| generatedFileName: fileName, | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * Get all student ids of students who have one or more updates | ||
| * between the given period. | ||
| * The updates can be one or more of the following: | ||
| * - Student or User data | ||
| * - SIN validation data | ||
| * - CAS supplier data | ||
| * - Overawards data | ||
| * @param modifiedSince the date after which the student data was updated. | ||
| * @param modifiedUntil the date until which the student data was updated. | ||
| */ | ||
| async getAllStudentsWithUpdates( | ||
| modifiedSince: Date, | ||
| modifiedUntil: Date, | ||
| ): Promise<number[]> { | ||
| const studentIdAlias = "studentId"; | ||
| const applicationsWithStudentUpdates = await this.applicationRepo | ||
| .createQueryBuilder("application") | ||
| .select("student.id", studentIdAlias) | ||
| .distinctOn([`"${studentIdAlias}"`]) | ||
| .innerJoin("application.student", "student") | ||
| .innerJoin("student.user", "user") | ||
| .innerJoin("student.sinValidation", "sinValidation") | ||
| .innerJoin("student.casSupplier", "casSupplier") | ||
| .leftJoin("student.overawards", "overaward") | ||
| .where("application.applicationStatus != :overwritten") | ||
| .andWhere("application.currentAssessment is not null") | ||
| // Check if the student data was updated in the given period. | ||
| .andWhere( | ||
| new Brackets((qb) => { | ||
| qb.where( | ||
| "student.updatedAt > :modifiedSince AND student.updatedAt <= :modifiedUntil", | ||
| ) | ||
| .orWhere( | ||
| "user.updatedAt > :modifiedSince AND user.updatedAt <= :modifiedUntil", | ||
| ) | ||
| .orWhere( | ||
| "sinValidation.updatedAt > :modifiedSince AND sinValidation.updatedAt <= :modifiedUntil", | ||
| ) | ||
| .orWhere( | ||
| "casSupplier.updatedAt > :modifiedSince AND casSupplier.updatedAt <= :modifiedUntil", | ||
| ) | ||
| .orWhere( | ||
| "overaward.updatedAt > :modifiedSince AND overaward.updatedAt <= :modifiedUntil", | ||
| ); | ||
| }), | ||
| ) | ||
| .setParameters({ | ||
| overwritten: ApplicationStatus.Overwritten, | ||
| modifiedSince, | ||
| modifiedUntil, | ||
| }) | ||
| .getRawMany<{ [studentIdAlias]: number }>(); | ||
| // Extract the student ids from the applications. | ||
| const modifiedStudentIds = applicationsWithStudentUpdates.map( | ||
| (applicationWithStudentUpdate) => | ||
| applicationWithStudentUpdate[studentIdAlias], | ||
| ); | ||
|
|
||
| return modifiedStudentIds; | ||
| } | ||
|
|
||
| /** | ||
| * Get student details of students who have one or more updates. | ||
| * @param studentIds student ids. | ||
| * @returns student details. | ||
| */ | ||
| async getStudentRecordsByStudentIds( | ||
| studentIds: number[], | ||
| ): Promise<StudentDetail[]> { | ||
| const queryResult = await this.studentRepo | ||
| .createQueryBuilder("student") | ||
| .select([ | ||
| "student.id", | ||
| "student.birthDate", | ||
| "student.disabilityStatus", | ||
| "student.disabilityStatusEffectiveDate", | ||
| "user.firstName", | ||
| "user.lastName", | ||
| "sinValidation.sin", | ||
| "casSupplier.supplierNumber", | ||
| "casSupplier.supplierAddress", | ||
| ]) | ||
| .addSelect("SUM(cslfOveraward.overawardValue)", "cslfOverawardTotal") | ||
| .addSelect("SUM(bcslOveraward.overawardValue)", "bcslOverawardTotal") | ||
| .innerJoin("student.user", "user") | ||
| .innerJoin("student.sinValidation", "sinValidation") | ||
| .innerJoin("student.casSupplier", "casSupplier") | ||
| .leftJoin( | ||
| "student.overawards", | ||
| "cslfOveraward", | ||
| "cslfOveraward.disbursementValueCode = :cslfAwardCode", | ||
| ) | ||
| .leftJoin( | ||
| "student.overawards", | ||
| "bcslOveraward", | ||
| "bcslOveraward.disbursementValueCode = :bcslAwardCode", | ||
| ) | ||
| .groupBy("student.id") | ||
| .addGroupBy("user.id") | ||
| .addGroupBy("sinValidation.id") | ||
| .addGroupBy("casSupplier.id") | ||
| .where("student.id IN (:...studentIds)") | ||
| .setParameters({ | ||
| cslfAwardCode: CANADA_STUDENT_LOAN_FULL_TIME_AWARD_CODE, | ||
| bcslAwardCode: BC_STUDENT_LOAN_AWARD_CODE, | ||
| studentIds, | ||
| }) | ||
| .getRawAndEntities(); | ||
|
|
||
| return mapFromRawAndEntities<StudentDetail>( | ||
| queryResult, | ||
| "cslfOverawardTotal", | ||
| "bcslOverawardTotal", | ||
| ); | ||
| } | ||
| // TODO: SIMS to SFAS - Add methods to extract application and restriction data. | ||
| } |
Oops, something went wrong.