SSIS: ProTips - Performance Tuning

SQL Saturday - Aug 4, 2012 - Baton Rouge

Instructor: Tim Costello (Interwork - Dallas)

SSIS: ProTips - Performance Tuning

  1. Benchmark before you begin
    • Performance WireTap script task component written by Todd McDermid
    • output window show start and end row - to show the execution time
    • lightweight (even if left in when running in production, it is lightweight enough not to hurt overall performance)
  2. Its All About the Buffers
    • properties
    • recommend keep defaults (start with low hanging fruit first)
    • dont mess with buffers first - check other stuff for possibilities
    • better to have small # of large buffers, than large # of small buffers
    • Synchronous tranformations - VERY fast (same buffer from beginning to the end)
    • Asynchronous transformations (creates new buffer)
  3. Optimize your souces
    • remove unneeded columns
    • if pulling from a database, use the SQL command (your own query) rather than the table name in the drop down - there is a BIG benefit to this
  4. Better tool for the job?
    • is SSIS the right tool for what you need?
    • if going from a database to a database - do it in the database, not SSIS

Other things discussed: 3 types of data flow components:

  • Full-blocking transformations
  • Semi-blocking transformations
  • Non-blocking transformations

Oh where, Oh where ... has my column gone?

Phew - I ran into something I knew would be soooo simple, but couldnt figure it out!  Don't you hate that?  Luckily a co-worker helped plug away at it and wham-o we found it!

I had a derived column with a little bit of logic, but not much, then added a data viewer after it to see the results.  Then, added another column within the derived column step.  When I went to look at the results in the data viewer - my column was no where to be found!  Ohh my!!

Turns out, when you create a data viewer it takes the columns it gets, and if you add another upstream, it wont reflect that change.  To change this, you go to the Data Flow Path Editor and click on the "Configure..." button in the bottom right corner (SSIS 2008 R2).  When you bring up the configuration dialog window, you will see a list of columns.  More than likely your new column is in the left hand box - move it to the right and presto! - you can now view that data to your heart's content!  What my co-worker pointed out was that you can also remove the extraneous columns you dont care to look at when you are using the data viewer.

Hope this helps someone find that missing column! :)

Using Transactions

I wanted to jot some notes down about using transactions and @@trancount - mostly, I personally like checking for if I should begin a transaction or not at the beginning of stored procedures.  I know this comes from me being a software developer and thinking the database guy/gal handles all that, and then learning more about the database side of things and realizing database coding isnt always reviewed that closely by a DBA.  So, it came time for me to learn.  Now that I switched jobs, I dont have all the code blocks to copy/paste - so this blog post will help with that! :)

Lets start with the basics:

A transaction is a sequence of operations performed in it's own "bubble" from other processes and/or transactions occuring.  Once a transaction is completed, the effects are permanent in the database.

Now, that is a simple explanation, as with everything, there are many things that go into it - such as how transactions affect each other because the data that is modified in 1 transaction is the same data needed for another transaction, so the "bubble" a transaction is in isnt completely isolated - or how if a transaction is rolled back, there is no permanent effect because nothing was done, etc.  I am going to talk about some of the ways of starting, monitoring, and ending a transaction, as well as nested transactions, and possibly uncommitable transactions.

To start a transaction, the syntax is BEGIN TRANSACTION (or BEGIN TRAN).  No matter what, this is pretty much straight forward.  To end a transaction, you would either use COMMIT TRANSACTION, or ROLLBACK TRANSACTION, depending on if you want to make the changes you made, or not make the desired changes, respectively.  That, in and of itself, is all you need to start utilizing transactions now - but I will keep going with more juicy info. :)

T-SQL Random Number Generator

I recently needed to generate a random number.  I know using rand() was the way to go, but it wsnt giving me the formatting I needed.  I went online (lazy, cause I know I could format a stupid number) and found an amazing little script that I expanded upon.  It gives you a min and max range which was perfect!

Here is the code and a link to the site I got it from (giving props to Michelle at for it!)

Declare @maxRandomValue tinyint = 100
	, @minRandomValue tinyint = 0;
Select Cast(((@maxRandomValue + 1) - @minRandomValue) 
	* Rand() + @minRandomValue As tinyint) As 'randomNumber';


I was wary of XML when I first heard about it - mostly because I was still trying to wrap my head around other stuff that I was learning.  Therefore, I didnt learn much about it at first.  Then, I needed to pass multiple values to a stored procedure and looked into it.  I quickly fell in love!!  There are many posts about using XML within T-SQL.  I am posting one so that I can search on my own blog for this when I need it - I dont seem to keep it in my head long enough to use it the next time I need it. :)

I am going to use examples from my alerter system, that is where I am using it as I am writing this article - so 2 birds with 1 stone - cant beat that!  By the way, I will format the SQL code better - it looks weird being all gray!

T-SQL Database Alert/Mail System

I am working on a T-SQL "system" to send emails.  This will help so that in all the things that I use to send update emails, (ETL, sql jobs, etc.), I can just write a record in a table somewhere and the email will be handled for me.  Some benefits:

  • I can have the templates and recipients pre-defined so I dont have to worry about what to say. 
  • I dont have to worry about updating my SSIS packages or sql jobs when the text or recipients of the notification email needs to be changed. 
  • A history of the emails sent, to whom, and when (purely for taking up space, I am sure noone would doubt me if I said an email was for sure sent even though they didnt get it).
  • A queue that will allow the emails to be sent even if services are down (after they are recovered).

SSIS connecting to Oracle

Ohh boy - this is a biggie for me right now (actually it has been for a couple of weeks, but I have successfully put it off until today). :)

I have a situation where there are already SSIS packages, and sql statements that pull from an Oracle database into our SQL Server database.  The person who set them up left the company (a positive for him).  Therefore, I was scrambling when I started getting all these error messages that were basically laughing at me!

These are the steps I have had to do:

Thank you for visiting!