I need assistance pulling a big db into SQL Server

[MadPiet]

Where's the file? I wanna see if I can get it to insert.

 

[jwcolby54]

What is this? And how do I use it?

 

[jwcolby54]

I have a text file. It appears to be a csv file.

 

[jwcolby54]

I have an xml file

 

[jwcolby54]

I have a schema.ini

 

[jwcolby54]

What I don't have is it imported into SQ Server!

To use BCP I had to go into the ini and manually copy the ini field lengths into the BCP field mapping thingie. Thingie is a very scientific term I picked up somewhere.

 

[jwcolby54]

Having done that I successfully pulled in several million records out of (apparently) somewhere around 60 million addresses. I got some dat from around 20 states, many just a few thousand records. Some states with tens or hundreds of thousands of records. My guess is that those states were simply first in the file position wise.

 

[jwcolby54]

BCP was unable to handle the very common field data like

Peoria,,,"Country Meadows Units 5, 6, & 7",,,,,,AZ,

With field delimiters of "" and inside that comma delimited strings. Failed to import every record with such data and there were a ton of them. So BCP kinda worked but not well...

 

[jwcolby54]

Where's the file? I wanna see if I can get it to insert.

https://www.transportation.gov/gis/national-address-database

 

[jwcolby54]

https://www.transportation.gov/gis/national-address-database

It took me hours to download, although my internet down speed isn't particularly speedy.

 

[amorosik]

IIRC it is a 36 gb file. Compressed. However I might have it in a usable format already. Do you know what GDB files are?

Interbase / Firebird
If the file format was really Interbase/Firebird then you could use this db server directly and connect via odbc to the data
No conversion, no data loss, no license fee to pay, and an exceptional db server to use for every purpose

 

[jwcolby54]

Where's the file? I wanna see if I can get it to insert.

The file is a 36gb download for the text download. I can't just append it to a message here.

And that is zipped.

 

[jwcolby54]

Interbase / Firebird
If the file format was really Interbase/Firebird then you could use this db server directly and connect via odbc to the data
No conversion, no data loss, no license fee to pay, and an exceptional db server to use for every purpose

I think this is a geo database, not the one I was trying to import.