Pragmatic Works Blog

I Can Upsert 700x Faster Than You!

Written by Pragmatic Works | Jun 16, 2017

Would you like to upsert (conditionally insert and update data) faster? How about 700x faster? Our data analyst, Achara McNair-Tarfa claims that she can upsert 700x faster than you. How does she do this? Pragmatic Works Sr. Product Engineer, Manuel Quintana, shows you how in this demo using a component in our Task Factory product.

Using Visual Studio, the traditional method for performing updates on a package is you have a source, it’s leveraging a lookup and you are effectively splitting out your inserts, version and updates.  This also performs updates in the data flow of SSIS, which means everything will occur row-by-row, which can be very slow. This is common practice, but there’s a better way.

With our Task Factory component, called Upsert Destination, you can skip that step. Within the simple interface design you’ll see 3 steps. Your source, whatever data cleansing you need to do and into the Upsert Destination. In the editor, you designate what key column you want to use and the comparison is done from source to destination. If that key is present, you update and if it’s not, you do an insert. Update and insert are being done at the same time, which saves you time right there.

But what actually allows you to go 700x faster? The answer is simple – the Bulk Method. With Task Factory’s Upsert Destination, you can process inserts and updates in batches of 10,000 rows instead of row by row. This efficient method can be used for updating anything from your data warehouse to a table.

Upsert Destination is only one of the over 60 components in Task Factory that will save you time and make you more productive and efficient. And Task Factory is forever free in Visual Studio. Visit our website or click below to learn more about making your upserts 700x faster, as well as many other incredible benefits for you and your business.

Try Task Factory for Free