De-dupe

[davesmith202]

I have about 2,000 records, each of which contains a field called Keyword.

In a text file, I have about 600 rows of text, normally 2 or 3 words on each row.

The text file are new keywords and I want to remove the duplicates from there.

How can I de-dupe one against the other?

Thanks,

Dave

 

[modest]

1) Choose one of the three methods:
a) Import the keywords from the file and run a delete query on the table (deleting duplicates)
b) Import the keywords from the file then Create a new table from that one, selecting distinct, or using group by
c) Create your table and make the keyword field a primary key (this will automatically make sure you can don't copy the same thing multiple times)


I would choose option (c) or (b)

 

[davesmith202]

I need to have list A, that is already entered into the database, stay as it is. But the text file containing the new keywords needs to have the duplicates removed. i.e. you are left with a text file which has just the new keywords and the ones already in the database removed.

Would (a), (b) or (c) work under those circumstances?

 

[modest]

Yes. Just post your files and I'll show you what I mean.

 

[davesmith202]

Unfortunately, the database is full of confidential data and the data is in a MySQL backend.

Any chance in just explaining the generic principle or just doing a sample based on one table with one field called Keyword?

Are you suggesting importing the text file into a table, then running the existing Keyword table against the newly imported Keyword table using the delete query and the end result is a table with just the new keywords? And then export back to the text file?

 

[modest]

it's an easier to understand algorithm if you just import, do all your data manipulation and then export