Update csvs_convert. #736
Merged
Update csvs_convert. #736
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The main change is adding --dump option so that you can produce a dump file that will work with `psql` for postgres and `sqlite3` for sqlite. This uses the native import functions '\copy` for postgres (which allows remote copy) and `.import` for sqlite, which is the fastest way to insert into sqlite. This dumpfile is also useful to just inspect what the command will produce if you run it directly against a database. This is to addresses partially dathere#687 as it is the representation that most replicates what the command will do. Other changed include making sure strings that start with 0 (but are not 0) are treated as strings not numbers which should resolve dathere#688
|
Thanks @kindly! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main change is adding --dump option so that you can produce a dump file that will work with
psqlfor postgres andsqlite3for sqlite.This uses the native import functions '\copy
for postgres (which allows remote copy) and.import` for sqlite, which is the fastest way to insert into sqlite.This dumpfile is also useful to just
inspect what the command will produce if you run it directly against a database. This is to addresses partially
#687 as it is the representation that most replicates what the command will do.
Other change includes making sure strings that start with 0 (but are not 0) are treated as strings not numbers which should resolve #688
Also better parquet date detection has been added resolving #697