You didnt supply the DDL, so I can only point to existing practices to
remove duplicate records.Here they are:
INF: How to Remove Duplicate Rows From a Table
INF: Removing Duplicate Rows from an Existing Table in SQL
Removing Duplicate Records
>>>> Should I use primary keys to prevent this?
Yes or a UNIQUE constraint depending upon the business logic.
>>>Would the job fail or just not load the dups?
May be you can load it first to a temp table and then use the logic
mentioned in the above articles before inserting to the main ...