-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add CSV Import File Validations #1211
Changes from 2 commits
a824103
e0e5f86
53d1ae7
50ecffc
e922cd8
e93623c
3911c0e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,42 +13,57 @@ def initialize(file) | |
end | ||
|
||
def call | ||
CSV.foreach(@file.to_path, headers: true, skip_blanks: true).with_index(1) do |row, index| | ||
# Using Google Form headers | ||
email = row["Email"].downcase | ||
csv_timestamp = Time.parse(row["Timestamp"]) | ||
catch(:halt_import) do | ||
validate_file | ||
|
||
person = Person.find_by(email:, organization: @organization) | ||
previously_matched_form_submission = FormSubmission.where(person:, csv_timestamp:) | ||
CSV.foreach(@file.to_path, headers: true, skip_blanks: true).with_index(1) do |row, index| | ||
# Using Google Form headers | ||
email = row["Email"].downcase | ||
csv_timestamp = Time.parse(row["Timestamp"]) | ||
|
||
if person.nil? | ||
@no_match << [index, email] | ||
elsif previously_matched_form_submission.present? | ||
next | ||
else | ||
latest_form_submission = person.latest_form_submission | ||
ActiveRecord::Base.transaction do | ||
# This checks for the empty form submission that is added when a person is created | ||
if latest_form_submission.csv_timestamp.nil? && latest_form_submission.form_answers.empty? | ||
latest_form_submission.update!(csv_timestamp:) | ||
create_form_answers(latest_form_submission, row) | ||
else | ||
# if the person submits a new/updated form, | ||
# i.e. an additional row in the csv with the same email / different timestamp, | ||
# a new form_submission will be created | ||
create_form_answers(FormSubmission.create!(person:, csv_timestamp:), row) | ||
person = Person.find_by(email:, organization: @organization) | ||
previously_matched_form_submission = FormSubmission.where(person:, csv_timestamp:) | ||
|
||
if person.nil? | ||
@no_match << [index, email] | ||
elsif previously_matched_form_submission.present? | ||
next | ||
else | ||
latest_form_submission = person.latest_form_submission | ||
ActiveRecord::Base.transaction do | ||
# This checks for the empty form submission that is added when a person is created | ||
if latest_form_submission.csv_timestamp.nil? && latest_form_submission.form_answers.empty? | ||
latest_form_submission.update!(csv_timestamp:) | ||
create_form_answers(latest_form_submission, row) | ||
else | ||
# if the person submits a new/updated form, | ||
# i.e. an additional row in the csv with the same email / different timestamp, | ||
# a new form_submission will be created | ||
create_form_answers(FormSubmission.create!(person:, csv_timestamp:), row) | ||
end | ||
@count += 1 | ||
end | ||
@count += 1 | ||
end | ||
rescue => e | ||
@errors << [index, e] | ||
end | ||
rescue => e | ||
@errors << [index, e] | ||
end | ||
Status.new(@errors.empty?, @count, @no_match, @errors) | ||
end | ||
|
||
private | ||
|
||
def validate_file | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the vein of single responsibility, we could have separate validation methods for each validation. Might be something to consider if we add more validations. I am fine either way in this case. |
||
raise FileTypeError unless @file.content_type == "text/csv" | ||
|
||
first_row = CSV.foreach(@file.to_path).first | ||
raise EmailColumnError if first_row.nil? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If I am understanding this correctly, we are raising There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes it is checking for an empty file as an edge case (came up during testing). I just reused the same error because technically the file does not have the "Email" column but I can add a more specific error. |
||
raise EmailColumnError unless first_row.include?("Email") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. shall we add lowercase There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I'm confused on what you are asking. If the header row doesn't include "Email"(capitalized) the error is raised. The service is currently using "Email". Do you want "email" to work as well? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry yeh that’s right. We may as well allow Email and email as valid because users won’t adhere to case rules half the time. |
||
rescue EmailColumnError, FileTypeError => e | ||
@errors << e | ||
throw :halt_import | ||
end | ||
|
||
def create_form_answers(form_submission, row) | ||
row.each do |col| | ||
next if col[0] == "Email" || col[0] == "Timestamp" | ||
|
@@ -61,6 +76,18 @@ def create_form_answers(form_submission, row) | |
) | ||
end | ||
end | ||
|
||
class EmailColumnError < StandardError | ||
def message | ||
'The column header "Email" was not found in the attached csv' | ||
end | ||
end | ||
|
||
class FileTypeError < StandardError | ||
def message | ||
"Invalid File Type: File type must be CSV" | ||
end | ||
end | ||
end | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! I like the structure.