Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect schema parameter is passed to org.neo4j.etl.NeoIntegrationCli #72

Open
RichardMacaskill opened this issue Jun 24, 2020 · 6 comments

Comments

@RichardMacaskill
Copy link

Using the ETL mapping tool, under Neo4j Desktop v4.0.4, when I click Start Mapping I get a 'Mapping Error' warning.

Looking in the logs, I see the error

  • Command failed due to error (SQLException: No matching schemas found).

and I see the following has been invoked:
java -cp "/Users/cas/Library/Application Support/Neo4j Desktop/Application/graphApps/_global/neo4j-etl-ui/dist/neo4j-etl.jar:/Users/cas/Library/Application Support/JetBrains/IntelliJIdea2020.1/jdbc-drivers/SQL Server/7.4.1/mssql-jdbc-7.4.1.jre8.jar" org.neo4j.etl.NeoIntegrationCli generate-metadata-mapping --rdbms:url "jdbc:sqlserver://localhost:1433;databaseName=AdventureWorks" --rdbms:password "MYPASSSWORD" --rdbms:user "sa" --schema "SalesLT" --output-mapping-file "/var/folders/pd/1by7j2s53x5cmk0_f4lbphk80000gn/T/mssql_AdventureWorks_SalesLT_mapping.json"

It looks like the --schema parameter should have been --rdbms:schema. If I make that change, the command completes at the command line as expected.

@conker84
Copy link
Collaborator

conker84 commented Jul 2, 2020

Hi @RichardMacaskill we have --rdbms:schema, -s, --schema that are all synonyms. Is it possible that the schema SalesLT that you're looking for is not in the database AdventureWorks?
Looking at the schema:
https://i0.wp.com/improveandrepeat.com/wp-content/uploads/2018/12/AdvWorksOLTPSchemaVisio.png?ssl=1
you should leverage Sales and not SalesLT
Please lemme have your feedback!

@RichardMacaskill
Copy link
Author

Thanks @conker84 but I don't think that's the behaviour I'm seeing. MS distribute a lightweight version of AdventureWorks called AdventureWorksLT where the Sales schema is renamed to SalesLT; https://docs.microsoft.com/en-us/sql/samples/adventureworks-install-configure?view=sql-server-ver15&tabs=tsql

image

These are the exact outputs I'm seeing when running 2 commands, which are only differentiated by the --rdbms:schema notation in the latter as opposed to --schema in the former

➜ ~ java -cp "/Users/cas/Library/Application Support/Neo4j Desktop/Application/graphApps/_global/neo4j-etl-ui/dist/neo4j-etl.jar:/Users/cas/Library/Application Support/JetBrains/IntelliJIdea2020.1/jdbc-drivers/SQL Server/7.4.1/mssql-jdbc-7.4.1.jre8.jar" org.neo4j.etl.NeoIntegrationCli generate-metadata-mapping --rdbms:url "jdbc:sqlserver://localhost:1433;databaseName=AdventureWorks" --rdbms:password "PASS" --rdbms:user "sa" --schema "SalesLT" --output-mapping-file "/var/folders/pd/1by7j2s53x5cmk0_f4lbphk80000gn/T/mssql_AdventureWorks_SalesLT_mapping.json"

  • Skipping reading import options from file because file [] doesn't exist.

  • Creating RDBMS to CSV mappings...
    Available schema: AdventureWorks.SalesLT

  • Using database plugin for

  • Crawling schemas

  • Retrieving all schemas

  • Retrieving all catalogs

  • Processed 14 rows for

  • Retrieved 0 schemas

  • Total time taken for - 00:00:00.013 hours

  • 92.3% - 00:00:00.012 -

  • 0.0% - 00:00:00.000 -

  • Command failed due to error (SQLException: No matching schemas found). Rerun with --debug flag for detailed diagnostic information.

➜ ~ java -cp "/Users/cas/Library/Application Support/Neo4j Desktop/Application/graphApps/_global/neo4j-etl-ui/dist/neo4j-etl.jar:/Users/cas/Library/Application Support/JetBrains/IntelliJIdea2020.1/jdbc-drivers/SQL Server/7.4.1/mssql-jdbc-7.4.1.jre8.jar" org.neo4j.etl.NeoIntegrationCli generate-metadata-mapping --rdbms:url "jdbc:sqlserver://localhost:1433;databaseName=AdventureWorks" --rdbms:password "PASS" --rdbms:user "sa" —rdbms:schema "SalesLT" --output-mapping-file "/var/folders/pd/1by7j2s53x5cmk0_f4lbphk80000gn/T/mssql_AdventureWorks_SalesLT_mapping.json"

  • Skipping reading import options from file because file [] doesn't exist.

  • Creating RDBMS to CSV mappings...
    Available schema: AdventureWorks.SalesLT

  • Using database plugin for

  • Crawling schemas

  • Retrieving all schemas

  • Retrieving all catalogs

  • Processed 14 rows for

  • Including schema AdventureWorks.SalesLT

  • Retrieved 1 schemas

  • Including schema AdventureWorks.SalesLT

  • Total time taken for - 00:00:00.019 hours

  • 94.7% - 00:00:00.018 -

  • 0.0% - 00:00:00.000 -

  • Retrieving database information

  • Not retrieving additional database information, since this was not requested

  • Retrieving JDBC driver information

  • Not retrieving additional JDBC driver information, since this was not requested

  • Retrieving SchemaCrawler crawl information

  • Total time taken for - 00:00:00.006 hours

  • 50.0% - 00:00:00.003 -

  • 0.0% - 00:00:00.000 -

  • 16.7% - 00:00:00.001 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • Crawling column data types

  • Retrieving system column data types

  • Not retrieving user column data types, since this was not requested

  • Total time taken for - 00:00:00.036 hours
    -100.0% - 00:00:00.036 -

  • 0.0% - 00:00:00.000 -

  • Crawling tables

  • Retrieving tables

  • Retrieving tables for schema <AdventureWorks.SalesLT>

  • Processed 13 rows for

  • Retrieved 13 tables

  • Retrieving table columns

  • Retrieving foreign keys

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Including schema AdventureWorks.SalesLT

  • Retrieving primary keys and indexes

  • Retrieving indexes

  • Retrieving primary keys

  • Total time taken for - 00:00:00.422 hours

  • 7.6% - 00:00:00.032 -

  • 49.5% - 00:00:00.209 -

  • 21.3% - 00:00:00.090 -

  • 2.1% - 00:00:00.009 -

  • 19.0% - 00:00:00.080 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • 0.0% - 00:00:00.000 -

  • Crawling routines

  • Retrieved 0 routines

  • Not retrieving synonyms, since this was not requested

  • Not retrieving sequences, since this was not requested
    ➜ ~

@conker84
Copy link
Collaborator

conker84 commented Jul 8, 2020

Can you try by specifying the full schema AdventureWorksLT.SalesLT?

@conker84
Copy link
Collaborator

conker84 commented Jul 8, 2020

@RichardMacaskill please look at the comment above^^^

@RichardMacaskill
Copy link
Author

Hi @conker84 - AdventureWorksLT.SalesLT wouldn't refer to anything, the database is called AdventureWorks and the schema `SalesLT'.

However I tried the qualified parameter reference AdventureWorks.SalesLT and this completed successfully using both the --rdbms:schema notation AND using the --schema notation.

If I don't qualify the schema name with the database name (as I normally wouldn't when connected to a SQL Server database), it still works with the former and fails with the latter notation, as before.

I guess if the docs and UI were clear that the schema needs to be referenced with the database name, that would be a resolution to this issue.

@conker84
Copy link
Collaborator

conker84 commented Jul 9, 2020

@conker84 yes that was the name in my instance :)

We totally need to improve the docs because the full qualified name for the schema is needed only for MSSQL (it's a requirement of a library called SchemaCrawled that we use in order to standardize the DDL extraction from various RDBMS)

Thanks al lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants