It has been nearly a year since I started an extensive series about Extended Events. Previous to that start, I had already written a bunch of articles about Extended Events.

Among the batch of articles preceding the 60 day series one can find an article about shredding extended event data. Then as a part of the series (the first article in the 60 day series) one can find how to shred the action payload data attached to an extended event session.

You can read the article on shredding XEvent payload data here. Then you can follow that up by reading how to shred the Action data here. And once you are done with those and you really want to learn more about Extended Events, please read the whole series which can be found here.

All of that reading to catch up should keep you busy for a tad bit.

Upgrade

A year is a really long time to go without updating a script – according to some. Today, I have an update for both of the XML shredding scripts you just read about in the list of articles mentioned moments earlier in this post.

Before I dive into the script, let’s revisit a sample of the XML from an event session payload. Within an event session, you can see that there are nodes for both the data and the actions (highlighted in green and red). With all of this information found within the same XML for the session, it somewhat makes sense to try and parse all of the data at once.

In addition to parsing all of the XML for the data and the actions at the same time, it seems to also make sense to generate the statements that would parse the XML within a single effort. As you would have noted, my previous scripts were just that – scripts. That implies executing a separate script for each the data and the actions. Maybe it would make more sense to execute a single script.

It is that notion of a single script that constitutes this upgraded version of the script.

One Script

In the following script, I have tried to accomplish just that – a single script to create the entire XML parser for me, for you, and for anybody wishing to use it. I don’t want to have to remember the subtle nuances of how to parse each of the events each time I need to parse the session data. I want something that is quick, easy, and repeatable.

With all of that said, here is the script that I now use to parse my session data. You should notice that it has been simplified and is more extensive now.

DECLARE @EventName VARCHAR(64) = NULL --'query_post_execution_showplan'--'sp_statement_completed'
	,@ReadFlag VARCHAR(64) = 'readonly' --NULL if all columntypes are desired
	,@SessionName	VARCHAR(128) = 'AuditDeprecated' --'system_health' --NULL if all Sessions are desired --'Deadlock' --NULL
	,@UniqueEventColumns	TINYINT = 1 --1 for unique payload columns across all events within the session, NULL for all columns in session
	,@UniqueActions TINYINT = 1 --1 for unique actions, NULL for all actions in the session

/*gen tsql to parse session payload */
SELECT '/* parse the data */
IF EXISTS (SELECT OBJECT_ID(''tempdb.dbo.#xmlprocess''))
BEGIN
DROP TABLE #xmlprocess
END

SELECT CAST ([t2].[event_data] AS XML) AS event_data, t2.file_offset,t2.file_name, cte1.event_session_id--, '' AS event_predicate
	INTO #xmlprocess
	FROM ( SELECT REPLACE(CONVERT(NVARCHAR(128),sesf.value),''.xel'',''*.xel'') AS targetvalue, ses.event_session_id
				FROM sys.server_event_sessions ses
					INNER JOIN sys.server_event_session_fields sesf
						ON ses.event_session_id = sesf.event_session_id
					--INNER JOIN sys.server_event_session_events sese
					--	ON ses.event_session_id = sese.event_session_id
				WHERE sesf.name = ''filename''
					AND ses.name = ' + @SessionName +'
					) cte1
				OUTER APPLY sys.fn_xe_file_target_read_file(cte1.targetvalue,	NULL, NULL, NULL) t2
		;


SELECT x.event_data.value(''(event/@name)[1]'', ''varchar(50)'') AS event_name
		, x.event_data.value(''(event/@package)[1]'', ''varchar(50)'') AS package_name
		, DATEADD(hh, DATEDIFF(hh, GETUTCDATE(), CURRENT_TIMESTAMP),
					x.event_data.value(''(event/@timestamp)[1]'',
										''datetime2'')) AS [timestamp]';

WITH presel AS (
SELECT oc.OBJECT_NAME AS EventName
		,oc.name AS column_name, oc.type_name
		,',event_data.value(''(event/data[@name="' + oc.name + '"]/value)[1]'',''' + 
			CASE 
				WHEN ISNULL(xmv.name,'') = ''
					AND oc.type_name = 'guid'
				THEN 'uniqueidentifier'
				WHEN ISNULL(xmv.name,'') = ''
					AND oc.type_name = 'boolean'
				THEN 'bit'
				WHEN ISNULL(xmv.name,'') = ''
					AND oc.type_name = 'callstack'
				THEN 'varchar(max)'
				WHEN ISNULL(xmv.name,'') = ''
					AND oc.type_name <> 'unicode_string'
					AND oc.type_name <> 'ansi_string'
					AND oc.type_name <> 'ptr'
					AND oc.type_name NOT LIKE '%int%'
				THEN oc.type_name
				WHEN ISNULL(xmv.name,'') = ''
					AND oc.type_name LIKE '%int%'
				THEN 'bigint'
				ELSE 'varchar(max)' END + ''') AS ' + oc.name + '' AS ColumnXML
		,oc.column_type AS column_type
		,oc.column_value AS column_value
		,oc.description AS column_description
		,ca.map_value AS SearchKeyword
		,sesea.SessionName
		,rownum = ROW_NUMBER() OVER (PARTITION BY sesea.SessionName, oc.name ORDER BY oc.object_name)
	FROM sys.dm_xe_object_columns oc
	-- do we have any custom data types
		OUTER APPLY (SELECT DISTINCT mv.name FROM sys.dm_xe_map_values mv
			WHERE mv.name = oc.type_name
			AND mv.object_package_guid = oc.object_package_guid) xmv
	--just get the unique events that are tied to a session on the server (stopped or started state)
		CROSS APPLY (SELECT DISTINCT sese.name,ses.name AS SessionName
						FROM sys.server_event_session_events sese
							INNER JOIN sys.server_event_sessions ses
								ON sese.event_session_id = ses.event_session_id) sesea
	--keyword search phrase tied to the event
		CROSS APPLY (SELECT TOP 1 mv.map_value
						FROM sys.dm_xe_object_columns occ
						INNER JOIN sys.dm_xe_map_values mv
							ON occ.type_name = mv.name
							AND occ.column_value = mv.map_key
						WHERE occ.name = 'KEYWORD'
							AND occ.object_name = oc.object_name) ca
	WHERE oc.column_type <> @ReadFlag
		AND sesea.name = oc.object_name
		AND oc.object_name = ISNULL(@EventName,oc.object_name)
		AND sesea.SessionName = ISNULL(@SessionName,sesea.SessionName)
)

SELECT p.EventName
	 , p.column_name
	 , p.type_name
	 , p.ColumnXML
	 , p.column_type
	 , p.column_value
	 , p.column_description
	 , p.SearchKeyword
	 , p.SessionName
	FROM presel p
	WHERE p.rownum = ISNULL(@UniqueEventColumns,p.rownum)
	ORDER BY p.SessionName,p.EventName
	;

/* auto generate the xml associated to an event session action deployed to the server */
WITH presel AS (
	SELECT p.name AS package_name
	        ,o.name AS action_name
			,',event_data.value(''(event/action[@name="' + esa.name + '"]/value)[1]'', ''' + 
			CASE 
				WHEN o.type_name = 'guid'
				THEN 'uniqueidentifier'
				WHEN o.type_name = 'boolean'
				THEN 'bit'
				WHEN o.type_name = 'binary_data'
				THEN 'varbinary(max)'
				WHEN o.type_name = 'callstack'
				THEN 'varbinary(max)'
				WHEN o.type_name = 'filetime'
				THEN 'varbinary(max)'
				WHEN o.type_name = 'cpu_cycle'
				THEN 'varbinary(max)'
				WHEN ISNULL(o.type_name,'') = ''
				THEN NULL
				WHEN o.type_name <> 'unicode_string'
					AND o.type_name <> 'ansi_string'
					AND o.type_name <> 'ptr'
					AND o.type_name NOT LIKE '%int%'
				THEN o.type_name
				WHEN o.type_name LIKE '%int%'
				THEN 'int'
				ELSE 'varchar(max)' END + ''') AS ' + esa.name +'' AS ActionXML
			,ses.name AS EventSessionName
			, ese.name AS EventName
	        ,o.description
			, rownum = ROW_NUMBER() OVER (PARTITION BY ses.name,o.name ORDER BY ese.name)
	FROM sys.dm_xe_packages AS p
		INNER JOIN sys.dm_xe_objects AS o
			ON p.guid = o.package_guid
		INNER JOIN sys.server_event_session_actions esa
			ON o.name = esa.name
		INNER JOIN sys.server_event_sessions ses
			ON esa.event_session_id = ses.event_session_id
		INNER JOIN sys.server_event_session_events ese
			ON esa.event_session_id = ese.event_session_id
			AND ese.event_id = esa.event_id
	WHERE o.object_type = 'action'
		AND (o.capabilities IS NULL OR o.capabilities & 1 = 0)
		AND (p.capabilities IS NULL OR p.capabilities & 1 = 0)
		AND ese.name = ISNULL(@EventName,ese.name)
		AND ses.name = ISNULL(@SessionName,ses.name)
)
SELECT p.package_name
	 , p.action_name
	 , p.ActionXML
	 , p.EventSessionName
	 , p.EventName
	 , p.description
	FROM presel p
	WHERE p.rownum = ISNULL(@UniqueActions,p.rownum)
	ORDER BY p.EventSessionName, p.action_name, p.EventName
	;

/* gen last piece of xml parsing sql code block*/
SELECT '	FROM #xmlprocess x
		LEFT OUTER JOIN sys.server_event_session_events sese
			ON x.event_data.value(''(event/@name)[1]'', ''varchar(50)'') = sese.name
				AND x.event_session_id = sese.event_session_id
	ORDER BY timestamp
		, event_data.value(''(event/action[@name="event_sequence"]/value)[1]'',
							''varchar(max)'');'

Not only does this script slice and dice for you…Wait that’s not quite right.

This script doesn’t just generate the XML shredding strings for the payload and action data. This script also will produce the rest of the tsql statements that should precede and conclude the XML parsing statements. Yes it uses dynamic sql. That is to your advantage in this case.

Could there be other enhancements? Absolutely! More will be coming.

Enjoy this script and happy parsing.

By the time this post is published I will likely be pulling into the parking lot at the Subway World Headquarters, ready to receive my employee badge. My time with Ascension Information Services, and possibly healthcare IT, has come to an end....

The post Choosing Subway appeared first on SQL Hammer.

I use impersonation on a regular basis for a number of different tasks. This one is pretty basic but can make a big difference in how you handle security. Let’s use a very basic task as an example.

A developer wants to be able to truncate a table.

This isn’t an unreasonable request right? She’s writing a piece of code that loads a bunch of data into a staging table. She want’s to be able to truncate the table once the load is over. Unfortunately the permission required to do this is ALTER on the table. That’s not just going to let her truncate the table, it’s going to let her change the structure of the table. Not acceptable in a production environment. A development environment sure. Not a production one. So what do we do?

We use impersonation.

We are going to create a stored procedure to do the truncate for her. One of the options when creating a stored procedure is the EXECUTE AS clause that causes the sp to run as if another user had actually run it. We proceed one of two ways.

One option is to:

1. Create a new user, generally SQL Server authenticated, just for this purpose.
2. Grant the user alter permissions on the table to be truncated.
3. Write a stored procedure that does the truncate on the table.
4. Use the EXECUTE AS ‘UserName’ clause in the stored procedure.
5. Grant EXECUTE permissions on the stored procedure to the user that needs to truncate the table.

 
Now when the stored procedure runs it will run as if it’s UserName and has UserName’s permissions. Therefore it will be able to truncate the table. UserName can also alter the table, but we aren’t sharing UserName’s password with our devs.

The other way, simpler, and in some ways more secure, is to use EXECUTE AS OWNER. As long as the owner of the schema of the stored procedure and the owner of the schema of the table are the same then no explicit permissions or special user are required. To explain that another way when we use EXECUTE AS OWNER the stored procedure runs as if it was the owner of the stored procedure. So if the stored procedure is in the dbo schema, that happens to be owned by dbo the databases owner, then the stored procedure is run as if it was being run by the database owner.

That doesn’t sound overly safe does it? It helps if you understand that the only people who can create a stored procedure that impersonates another user are the people who have the ability to impersonate that user. So in this case the stored procedure will probably be created by a DBA who is a member of db_owner or sysadmin. That DBA can control exactly what the stored procedure does, so there is no more risk than anything else a DBA can do.

So here is an example of the stored procedure we have been talking about:

CREATE PROCEDURE dbo.Truncate_UserTest 
WITH EXECUTE AS OWNER
AS
TRUNCATE TABLE UserTest;
GO
GRANT EXECUTE ON Truncate_UserTest TO DevWhoNeedsToTruncate;

I’ve been spending some time lately working with statistics, specifically auto-created statistics. When auto-create statistics is enabled, SQL Server will create column statistics when it needs them. No surprises there. Conventional wisdom also states that SQL Server never creates multi-column statistics, only single column statistics. In fact, this MSDN page says this of the auto create statistics option:

“It applies strictly to single-column statistics for the full table.”

Seems pretty clear. SQL Server will not automatically create multi-column statistics. I’ve heard this over and over from many different sources.

So imagine my surprise when, while testing some code, I came upon entries in the sys.stats_column system table that looked like this:

Wow.. That sure looks like three auto-created, multi-column statistics! We have three stats: stats_ids 3, 4, and 5. The sys.stats_column table contains one row for each column that is in a statistic, so multiple rows for a single statistic (i.e., a single stats_id value), indicate multiple columns in that stat. Indeed, the column_id values indicate the table columns contained the stat. So stats_id 3 contains columns VersionMajor and ApplicationID (column_ids 3 and 1), stats_id 4 contains columns VersionMinor and ApplicationID (column_ids 4 and 1), and stats_id 5 contains columns VersionRevision and ApplicationID (column_ids 5 and 1). And, clearly, the auto_created flag is true, so these three stats were auto-created. What’s going on?

I puzzled over this for quite some time got nowhere. I finally reached out to the SQL Server community for help. Erin Stellato (b | t) said she had never seen that before, but she did mention the naming of the statistics seemed strange. I noticed this as well. As Paul Randal has blogged, auto-created statistics are named as _WA_Sys_<hex value of column>_<hex value of table>. Typically, the names look something like _WA_Sys_0000000D_00000029.

But these stats don’t follow that format. Instead of the hex value for the column, they have the column name.

A couple hours later, Paul White (b | t) responded to my tweet:

Aha! It seemed highly likely that this was the cause of what I was seeing. Although I was working with a database on a SQL 2012 server, I was pretty sure that database was originally created on an older version of SQL. I wanted to confirm this however.

To verify what SQL version the database was created with, I used the DBCC DBINFO command. The value of the dbi_createVersion field indicates what SQL version was used to create the database.

Thanks to Paul Randal’s TechNet post, we know that a value of 539 means the database was created with SQL 2000. Looks like Paul White might be on to something.

The next piece I wanted to verify was that auto create stats for SQL 2000 included the table’s clustering key. Unfortunately, deep dive details for auto-create stats for SQL 2000 are somewhat hard to come by these days and I was not able to find anything that definitely stated the clustering key was included. However, in all the auto-created stats in all the old databases I found on my servers, this was the case. I also verified that if the clustering key was more than one column, all the clustering columns were included in the auto created stats. I’m going to go ahead and assume this is the way it worked back in the day.

And finally, I wanted to find something confirming the naming convention. Again, details were hard to find, but I did find this ebook which states that the names of the auto-created statistics in SQL 2000 use the column name, not the hex value of the column.

This is enough evidence for me, so I’m going to call it: I’m going to go ahead and say that auto created column statistics can be created on multiple columns. Granted, you have to be running SQL 2000 and the additional columns are always the clustering key columns, so it’s very specific case and it no longer happen in modern versions of SQL Server.

Still, if you are ever asked if SQL Server will automatically create multi-column statistics, you can now give the classic SQL Server answer: It depends :-)

If SQL doesn’t behave this way anymore, why is this important? This issue caught my eye when I was writing some code to examine the statistics tables and when I joined to the sys.stats_columns table, I was getting back more rows than I expected. Because I was joining to other tables as well, this messed up my final result set. If you’ve got databases that have been migrated from older versions of SQL, you might run into this problem as well. Come back next week for the story on that…

P.S. Just for kicks, I deleted one of the old, two column stats and ran a query so that a new stat would be created. Would the new statistic follow the old or new SQL Server rule? Turns out, not surprisingly, the newly auto-created stat followed the new rules: it was single column, did not include the clustering key, and followed the new naming convention using the hex value for the column name. This was tested using SQL 2012.

I also wanted to see what would happen if the database was running in SQL 2000 compatibility mode. On a SQL 2005 server, I found a database that was created in SQL 2000 and had some of these strange auto-created stats. I changed the database to SQL 2000 compatibility mode, deleted one of the auto-created stats, and ran a query to force a new one to be created. The result? A single column stat that did not include the clustering key and that was named using the hex id of the column. In other words, the stat was made using the method we see in newer versions of SQL Server. So the database compatibility mode does not affect this. The only way multi-column auto created stats can be generated is if you are using the SQL Server 2000 (or earlier, most likely) database engine.

It’s Monday and time for this week’s blog and twitter round-up for last week. If you haven’t already, follow me on twitter (@StrateSQL). This is a good chance to catch up on data platform technology and career related information I’ve shared in the last week and activity on this blog.

Most Popular Article Shared

Last weeks most popular link is a blog post from Steve Jones (blog | @) on R Services and a white paper written by Joe Sack.  While the meat of the content is the white paper reference, Steve does talk a bit about what we need to see more of when it comes to R.  Check out both, the white paper can help get you going and Steve’s post provides ideas on sharing your knowledge.

http://mobile.twitter.com/StrateSQL/status/757980829381488641

Last Week’s Popular Posts

The most popular posts on this blog in the week are:

1. 31 Days of SSIS – The Introduction (759)
2. 31 Days of SSIS – Raw Files Are Awesome (1/31) (347)
3. 31 Days of SSIS as a Book (287)
4. Get Just The Tools: SSMS Download (227)
5. XQuery for the Non-Expert – Value (120)
6. Security Questions: Removing Logins From Databases (119)
7. 31 Days of SSIS – Using Breakpoints (2/31) (110)
8. 31 Days of SSIS – Generating Row Numbers (23/31) (109)
9. Determining Filegroup for a Table (80)
10. Security Questions: Difference Between db_datawriter and db_ddladmin? (76)

Last Week’s Top 20 “Reading” Links

Along with the most popular link, here are the top twenty items relating to SQL Server, technology and careers that were shared last week. If you missed them throughout the week, here’s the opportunity to get caught up on some items that other’s read after I linked them out.

1. Why Use R Services? [24 clicks]
2. Certification Update – SQL 2016 [21 clicks]
3. Slimmer SQL Saturdays [18 clicks]
4. MapR, Microsoft make announcements at Hadoop and Red Hat summits [16 clicks]
5. SQL PowerShell: July 2016 update [13 clicks]
6. How is performance impacted by having too many NUMA nodes? [13 clicks]
7. Stretch Database in SQL Server 2016 RTM [12 clicks]
8. PASS Summit 2016 I’m Speaking [12 clicks]
9. Will Query Store Work in a Read Only Database? [12 clicks]
10. Summit Speakers Announced [10 clicks]
11. What Happens to Pages in the Buffer Pool when your Availablity Group Fails Over? [10 clicks]
12. Learn Index Tuning at the PASS Summit in 2016! [9 clicks]
13. Increased Memory Size for In-Memory OLTP in SQL Server 2016 [9 clicks]
14. Fail Over, Fail Again, Fail Better – Preparing for Disaster Recovery (Dear SQL DBA) [9 clicks]
15. Getting the Previous Row Value before SQL Server 2012 [9 clicks]
16. SQL Server 2016 Upgrade Testing with the New Cardinality Estimator: Context Matters [8 clicks]
17. Help, my company won’t pay for training! [8 clicks]
18. Everything?!? You Need to Know About Beta Registration [8 clicks]
19. Setting CE TraceFlags on a query-by-query (or session) basis [7 clicks]
20. LAST_VALUE–The Basics of Framing [7 clicks]

Last Week’s Posts From Previous Years

Sometimes the most useful content on a blog wasn’t written in the past week, it’s often other articles shared in the past that resonate with readers. Check out the following links that I published in past years over the past week:

1. Security Questions: Recommended Security Book (2013-07-27)
2. Gotta Stop Violating Foreign Keys (2009-07-29)
3. Finding Number of Reports Executed By Folder (2009-07-30)
4. Analyzing Report Execution Time (2009-07-31)

Other Items Shared

Of course, no week would be complete without a few off-topic links. These have nothing to do with technology or your career, but they are interesting and worth a second look.

1. Credit card companies are blowing it with chip payments [15 clicks]
2. WONDER WOMAN – Official Comic-Con Trailer (2017) Gal Gadot DC Superhero Movie [14 clicks]
3. Why Did Busloads Of Asian Tourists Suddenly Arrive In This English Village? [14 clicks]
4. What Really Divides American Politics: 2 Different Ideas of Freedom [13 clicks]
5. Elon Musk to Open His Tesla Gigafactory Years Ahead of Schedule [11 clicks]
6. These 9 Slides Put the New Tesla Gigafactory in Perspective [10 clicks]
7. DOCTOR STRANGE Trailer from SDCC Is a Big Leap from the MCU Look | Nerdist [10 clicks]
8. Microsoft Surpasses 1 Million Course Enrollments on edX Platform [9 clicks]
9. Science Says 92 Percent of People Don’t Achieve Their Goals. Here’s How the Other 8 Percent Do [8 clicks]
10. Blair Witch Is Back To Scare The Bejesus Out Of You [8 clicks]
11. Another Democratic Party committee has been hacked [8 clicks]
12. JACK REACHER: NEVER GO BACK Official Trailer (2016) Tom Cruise Movie [7 clicks]
13. SUICIDE SQUAD – Official Comic-Con Trailer (2016) DC Superhero Movie HD [7 clicks]
14. The problem with complaining about the system [7 clicks]
15. Marvel decides not to split Avengers: Infinity War into two movies [7 clicks]
16. Larry the Lobster Rescued From Restaurant, Dies Getting Shipped to Aquarium [7 clicks]
17. Tower of Terror to get superhero makeover at Disney California Adventure Park [6 clicks]
18. Guardians of the Galaxy Vol. 2 [6 clicks]
19. KONG: SKULL ISLAND – Official Comic-Con Trailer (2017) Tom Hiddleston Monster Movie [6 clicks]
20. Tesla Gigafactory launch event tonight will be streamed online [6 clicks]

This may come as shocking news to some, but Microsoft SQL Server and Microsoft Azure SQL DB (Azure SQL Database, Azure SQL DB, SQL Azure Database, or whatever name you know it as) are very much not the same thing.  They may look similar and they may act similar, but they are very much not the same thing. The same applies to other cloud providers who are offering SQL Server in a Platform as a Service (PaaS) offering such as Amazon’s AWS. They aren’t SQL Server and they aren’t Azure SQL, they are somewhere in between. While other cloud providers are just running a managed SQL Server implementation, they aren’t exposing the full SQL Server instance to you so various features won’t be available to you in their PaaS solutions.

As an administrator it’s very important to remember this as planning migrations and designing systems to run on one or the other have VERY different requirements . This is because Microsoft has committed to a Cloud First deployment method where the cloud is always going to be ahead of “the box” (what we install on-premises).

The code behind Microsoft SQL Server and Azure SQL DB is actually very similar. In fact SQL DB is a fork of the SQL Server code, and as new features are written into Azure SQL DB those code changes get merged back into the SQL Server codebase for release in the next service pack or the next major version of the SQL Server product.

The code that’s now included in SQL Server 2016 has been running in Azure SQL DB for months and many of the features which were “introduced” in SQL Server 2016 such as Row Level Security, Dynamic Data Masking, Replica Seeding, etc. were all in use in Azure SQL DB well before they were introduced to the on-premises product.

Because Microsoft is releasing features to the cloud first, this means that treating PaaS services like they are just an extension of the on-premises environment just doesn’t work.

Azure SQL DB for DR

For example, if you wanted to run a SQL Server on-premises and use Azure SQL DB for your DR strategy, my response would be that you’re trying to use the wrong technology.  Azure SQL DB and SQL Server are different beasts built to solve different problems.  The appropriate DR solution in Azure for a SQL Server running on-premises would be a virtual machine running in Azure IaaS  as the Azure IaaS solution would be a like for like solution supporting all the same features. Attempting to use a PaaS solution as a Disaster Recovery solution for an on-premises SQL Server deployment would mean that your DR solution isn’t the same as your production solution with features not being available. While you may not use any of those features today, can you guarantee that 5 years from now you still won’t be using those features?

Moving large dev databases to SQL DB

If you have a large database that you want to move to Azure SQL DB for testing, then you probably don’t want to be running that in Azure SQL DB to begin with.  You probably want to run that within a VM.  That means just uploading your backup to Azure, or if it’s too large for that just ship it to Microsoft using the import/export service.

If you are planning on using Azure SQL DB for your development platform, and SQL Server on-premises, you’re planning incorrectly. Because SQL DB is different from SQL Server the only development databases which should be hosted in SQL DB are databases which are hosted in SQL DB for production, or which are being moved to SQL DB.

One cloud for production, and one for DR

The thought of using multiple cloud providers for DR and expecting that to “just work” when using PaaS is problematic and naïve.  Suggesting this shows a lack of understanding about how PaaS services and the cloud work.  Each cloud provider has implemented their cloud solutions a little differently, so using one as DR for another isn’t going to be something which you can do right out of the box.  If you did want to put something like that together it would require that your application be written to handle that sort of workload, and error-handling using complex asynchronous messaging to ensure that databases in both cloud providers are up to date with current information.

Even when you are using PaaS a hybrid co-location provider to public cloud provider architecture is a better solution than trying to span public cloud providers. While there are very large firms who broker applications between clouds, this is difficult and requires advanced networking, automation, and coding skills.

Outgrowing SQL DB

There’s the worry that the database might outgrow the PaaS solution.  This is definitely a concern for those who are building services for PaaS platforms.  However, if you have designed sharding of your database correctly then this shouldn’t be an issue.  If you haven’t designed your sharding correctly, then your implementation will be a nightmare and it will need re-architecting in the future.  If you plan on simply moving monolithic databases from on-premises to a PaaS database platform, your migration will most likely fail. Back in the early days of Azure SQL Database this was a painful proposition (I believe the initial white paper was 150 pages). Now Microsoft offers a set of tools called elastic scale which greatly reduce any pain associated with sharding your database.

These are the among the reasons that you need to have the person who designs your cloud architecture to have a solid understanding of cloud and the limits of the various options available. Even then, you need someone who can think like an architect: documenting what your solution requirements are, understanding how those requirements are supported by a solution, and how to implement them in a way that works with your solution goals. If the person who’s designing your cloud migration doesn’t understand all the various solutions and options, then the migration is bound to have problems, many of which can be very complex to solve, taking a lot of time and potential a decent amount of outage time to resolve; not to mention wasted consulting dollars.

Not every database (or application) should be running in PaaS. Azure Database is best choice for apps that live in the cloud. If you are developing a new customer facing app, this is probably your best choice. In all likelihood you will never run your ERP system in a PaaS database. Knowing when to place systems in a PaaS solution, when to place systems in an IaaS solution, and when to place systems on-premises is the mark of a true solutions architect.

Denny

The post SQL Azure != SQL Server appeared first on SQL Server with Mr. Denny.

More of the same with the next Expert Performance Indexing for SQL Server book giveaway.  So far, I’ve sent out six copies of the book based on comments from the past six (1, 2, 3, 4, 5, and 6) posts.  I’ve made a small change to the title of the series (week vs. part), mainly because getting a kid off to college and summer vacation has been interfering with the rate of posts.  There will still be thirteen posts, though, going until they are all given away.

To re-cap from previous posts, Expert Performance Indexing for SQL Server is a book I co-wrote with Grant Fritchey (Blog | @gfritchey) last summer, which is a valuable resource to anyone building indexes on SQL Server.  As part of finishing the book, Apress sent me a number of copies of the book to share with people.  I figured the best way to share these out is to give them away to people that comment on this blog.

So here’s a topic for leaving a comment this week…

The Question

For part seven, the question will focus on chapter nine, which covers index maintenance.  The topic for discussion in the comments this time is…

Do you think that defragmenting indexes is worthwhile?  Why or why not?  What evidence do you have to support your position?

If you have a story or opinion to share, leave a comment below. After about a week (once I return from vacation), I’ll select one of the comments and send that author a copy of the book.

When working with SQL dates, sometimes you need to calculate the end of the month.  Months are tricky!  Some are 28 days, others 30 or 31, and now and then there’s a leap year!

So, given a date, how do you calculate the number of days remaining in the month?

The calculation is really a two-step process:

1. Determine the last date of the month.
2. Calculate the difference in days, between the date in question and the step 1.

We can use the DATEDIFF function do calculate the difference in days, but how do you calculate last day of the month?

Using EOMONTH to Calculate the End of the Month

In SQL Server 2012 and above, you can use the EOMONTH function to return the last day of the month.

For example

SELECT EOMONTH('02/04/2016')

Returns  02/29/2016

As you can see the EOMONTH function takes into account leap year.

So to calculate the number of day from a date to the end of the month you could write

SELECT DATEDIFF(d,'02/04/2016', EOMONTH('02/04/2016'))

Which returns 25.

Let’s try a more comprehensive example that calculates the days remaining on a LoanDate for the LoanDate’s current month:

BEGIN TRANSACTION
-- Sample Data
CREATE TABLE LoanDate (LoanID INT, LoanDate DATE);
INSERT INTO LoanDate Values (1, '1/1/2016');
INSERT INTO LoanDate Values (1, '1/15/2016');
INSERT INTO LoanDate Values (1, '1/31/2016');
INSERT INTO LoanDate Values (1, '2/15/2016');
INSERT INTO LoanDate Values (1, '3/15/2016');
INSERT INTO LoanDate Values (1, '4/15/2016');
INSERT INTO LoanDate Values (1, '5/15/2016');
INSERT INTO LoanDate Values (1, '6/15/2016');
INSERT INTO LoanDate Values (1, '7/15/2016');
INSERT INTO LoanDate Values (1, '8/15/2016');
INSERT INTO LoanDate Values (1, '9/15/2016');
INSERT INTO LoanDate Values (1, '10/15/2016');
INSERT INTO LoanDate Values (1, '11/15/2016');
INSERT INTO LoanDate Values (1, '12/15/2016');

-- Select LoanDate, Days in Month, and Days Remaining in Month
SELECT LoanID,
       LoanDate,
       EOMONTH(LoanDate) LoanDateEndOfMonth,
       DATEDIFF(d, LoanDate, EOMONTH(LoanDate)) as DaysRemaining
FROM   LoanDate
ROLLBACK

Here is the result:

As you can see, the DaysRemaining changes according to the month.  Also, notice that when the date occurs on the same date as the end of the month, such as row 3, zero days remain.

Now let’s look at how you would go about calculating this if you are using SQL 2008 R2 or before.

Old School Method to Calculate Last Day of Month

The calculation is really the same two-step process:

1. Determine the last date of the month
2. Calculate the difference in days, between the date in question and the step 1.

However, the difference is in how we determine the last date of the month.  Since EOMONTH isn’t available, we need to calculate it the “old fashioned” way.  There are several methods to do this.  Here is one.

We’ll calculate the last day of the month using two functions:  DATEADD and DAY.

We’ll use DATEADD to add a month to the date.  Then the DAY function to determine the number of days from the beginning of the month.  By subtracting this from the date we just calculated (the one that is a month ahead), we can get the last date of the month.

OK, I know that sounds confusing, so let’s break it down using this diagram:

Using the same example as we did with EOMONTH we have the following statement we can use to test:

BEGIN TRANSACTION
-- Sample Data
CREATE TABLE LoanDate (LoanID INT, LoanDate DATE);
INSERT INTO LoanDate Values (1, '1/1/2016');
INSERT INTO LoanDate Values (1, '1/15/2016');
INSERT INTO LoanDate Values (1, '1/31/2016');
INSERT INTO LoanDate Values (1, '2/15/2016');
INSERT INTO LoanDate Values (1, '3/15/2016');
INSERT INTO LoanDate Values (1, '4/15/2016');
INSERT INTO LoanDate Values (1, '5/15/2016');
INSERT INTO LoanDate Values (1, '6/15/2016');
INSERT INTO LoanDate Values (1, '7/15/2016');
INSERT INTO LoanDate Values (1, '8/15/2016');
INSERT INTO LoanDate Values (1, '9/15/2016');
INSERT INTO LoanDate Values (1, '10/15/2016');
INSERT INTO LoanDate Values (1, '11/15/2016');
INSERT INTO LoanDate Values (1, '12/15/2016');

-- Select LoanDate, Days in Month, and Days Remaining in Month
SELECT LoanID,
       LoanDate,
       DATEADD(dd,-(DAY(DATEADD(mm,1,LoanDate))), 
               DATEADD(mm,1,LoanDate)) LoanDateEndOfMonth,
       DATEDIFF(d,LoanDate, DATEADD(dd,-(DAY(DATEADD(mm,1,LoanDate))), 
                            DATEADD(mm,1,LoanDate))) as DaysRemaining
FROM   LoanDate
ROLLBACK

Here are the results:

So which method should you use?

If I’m writing code and I know will run on SQLServer 2012 or greater, I would go for the EOMONTH method as it is much easier to read; however, I think knowing and understanding the method using DATEADD and DAY help you better understand how to manipulate dates.

So what other ways have you see to calculate the last date in a month?  If would love to see them.  Just let me know in the comments below.

The post How Do You Calculate the End of the Month in SQL? appeared first on Essential SQL.

Recently I had a session around using SQL Server on Azure and during this session one of the questions was to access the SQL Server in a Hybrid Setup. This is a typical scenario wherein the developer was requesting to connect to the SQL Server running on a VM from his application that was sitting inside his environment. Though there were specific steps that needed to be taken, he was getting some error. Let us learn about how to setup firewall with Azure SQL Server VMs.

After a moment of debugging, I realized it was a problem of Firewall. Though the initial test was by disabling the Firewall I was able to get the connectivity working. I enabled it immediately and then started to get into configuring the Firewall for exception for SQL Server traffic. This blog is more of a tutorial on how I was configuring the same.

Steps to enable the SQL Server traffic

To enable connection to the SQL Server from on-premises applications, you must open port 1433 on the SQL Server VM. Though this is an example, you need to see if you can change and use some other port for SQL Server connection. But this is reserved for some other time. The following steps will lead you through this:

1. Login to SQL Server VM and Open firewall settings
2. Create an inbound rule for the TCP port 1433 to allow connections
   
   

1. Follow the default values on the wizard for the next steps and name the rule ‘SQL TCP’ and click OK.
   

Alternatively, you can use execute this PowerShell cmdlet to configure inbound firewall rule:

netsh advfirewall firewall add rule name='SQL Server (TCP-In)' program='C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Binn\sqlservr.exe' dir=in action=allow protocol=TCP 

This is an important step one needs to configure when working with VMs on Azure to access SQL Server. Though I have started to explore some of the core capabilities that come when working with Azure, I still see there are nuances that are quite different when compared to how an on-premise SQL Server would be. I am sure DBA’s and administrators are struggling to understand these use-cases that need to be configured because they have been used to connecting to SQL Server directly when working on their data centers.

Having said that, I felt this was something common and people are aware. But since I have done in my consulting couple of times in the past month that I get to write them down too.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Setting Firewall Settings With Azure SQL Server VMs

Up until August 1st if you had 2 vNets in the same Azure region (USWest for example) you needed to create a site to site VPN between them in order for the VMs within each vNet to be able to see each other.  I’m happy to report that this is no longer the case (it is still the default configuration).  On August 1st, 2016 Microsoft released a new version of the Azure portal which allows you to enable vNet peering between vNets within an account.

Now this feature is in public preview (aka. Beta) so you have to turn it on, which is done through Azure PowerShell. Thankfully it uses the Register-AzureRmProviderFeature cmdlet so you don’t need to have the newest Azure PowerShell installed, just something fairly recent (I have 1.0.7 installed). To enable the feature just request to be included in the beta like so (don’t forget to login with add-AzureRmAccount and then select-AzureRmSubscription).

Register-AzureRmProviderFeature -FeatureName AllowVnetPeering -ProviderNamespace Microsoft.Network –force

Now this “should” register you pretty quickly, but it took about an hour before I was able to actually setup a peer.  The first step is to check and see if you are registered or registering by using the script below.  If this says Registering then you have to wait.  If it says Registered then you should be good to go.  If you show as Registered and still get hours give it an hour then try again.

Get-AzureRmProviderFeature -FeatureName AllowVnetPeering -ProviderNamespace Microsoft.Network

To actually setup the peering you have to specify which vNets are allowed to talk to which other vNets.  The easiest way to do this is in the Azure Portal (unless you have a bunch to do then use PowerShell).  Log into the portal and navigate to your Virtual Networks.  In the properties of the vNet you’ll see a new option called “Peerings”.

Select this and click the “Add” button. Which will get you the new peering menu shown below.

Give the peer a name (I used the name of the vNet that I was connecting to), specify if the peering is to an RM or Classic vNet (yes you read that correctly, Classic is supported, do a degree) the subscription and the vNet that you want to connect to.  You can then enable and disable access to the vNet over the peer, and specify if the peer connection should allow forwarded traffic (from a site to site VPN for example) and if this peer should be allowed to use this vNets Gateway (if it has one).  If the vNet you’re configuring doesn’t have a remote network gateway, you can check that bottom button to use the gateway of the remote vNet instead.  Once that’s done click OK, then setup the same peering on the remove vNet.  Give it a minute or two for the deployments to complete, then you should have full private IP communications between the two vNets.

Now there’s a couple of restrictions to keep in mind.

1. This only works across vNets in the same account.
2. This only works across vNets in the same region, so vNets in different regions will still need to have a site to site VPN, but you only really need to have a single vNet in each region with a site to site VPN.
3. Classic is supported, sort of.  Classic vNets can ONLY peer to RM vNets, no Classic to Classic peering is supported.

That seems to be about it, or at least all I’ve run across.

Personally I think that this feature is fantastic, and it looks like it’ll solve one of my client’s issues that we’ve been working on for about a week now. I can’t wait to dive into it more and really push the limits.

Denny

The post Site to Site VPNs no longer needed for vNets in the same Azure region appeared first on SQL Server with Mr. Denny.

Following on from my last post on checking whether query folding is taking place in a Power BI/Power Query query, I’ve just stumbled on another way of doing this – by looking at metadata. I blogged about M metadata and Power Query some time ago (you might want to read the post to get some background), but it seems like metadata is used more widely than I thought in M…

Take the following query, as used in my previous blog post:

let
    Source = Sql.Database("localhost", "adventure works dw"),
    dbo_DimDate = Source{[Schema="dbo",Item="DimDate"]}[Data],
    #"Removed Other Columns" = 
    Table.SelectColumns(
        dbo_DimDate,
        {"DateKey", "FullDateAlternateKey", 
        "DayNumberOfWeek"}),
    #"Filtered Rows" = 
    Table.SelectRows(
        #"Removed Other Columns", 
        each ([DayNumberOfWeek] = 1))
in
    #"Filtered Rows"

As shown in that post this query connects to the Adventure Works DW database in SQL Server and gets a few columns plus some filtered rows from the DimDate table, and most importantly query folding takes place for all of these transformations.

It turns out that each of the variables used as steps in the query can have metadata automatically assigned to them, some of which is relevant to query folding, and we can use the Value.Metadata() function to get this metadata. So, taking the previous query, we can add two extra steps:

let
    Source = Sql.Database("localhost", "adventure works dw"),
    dbo_DimDate = Source{[Schema="dbo",Item="DimDate"]}[Data],
    #"Removed Other Columns" = 
    Table.SelectColumns(
        dbo_DimDate,
        {"DateKey", "FullDateAlternateKey", 
        "DayNumberOfWeek"}),
    #"Filtered Rows" = 
    Table.SelectRows(
        #"Removed Other Columns", 
        each ([DayNumberOfWeek] = 1)),
    GetMetadata = Value.Metadata(#"Filtered Rows"),
    QueryFolding = GetMetadata[QueryFolding]
in
    QueryFolding

The GetMetadata step gets the metadata associated with the #”Filtered Rows” step, and that metadata comes back in the form of a record with a single field called QueryFolding. The next step gets the data from that field, which is again in the form of a record:

So we can see quite clearly that this step is being folded and that SQL is being generated in the background. It doesn’t seem to be possible to get the SQL query generated here though.

One last thing to note: while all of this works for SQL Server, when I tested it on an SSAS data source where query folding is also definitely taking place the metadata told me – incorrectly – that there was no query folding happening (I couldn’t see the MDX query generated using the View Native Query menu option either). Maybe this is all still a work in progress?

Sharpen Something

Last week I sent out the invite for the August TSQL Tuesday blog party. In that invite I promised to send out a reminder seven days prior to the event. Well – it is that time.

You are cordially invited to read the invite for TSQL Tuesday 81 and plan your entry for the party.

In the invite, I shared the details for the event including examples of what I am looking for as an entry for the event.

I hope we will be seeing you next Tuesday, August 9th in attendance at this month’s party. I am sure it will prove to be an interesting experience one way or another.

Bonus Example

In the original invite I provided a list of examples of what one could do for this TSQL Tuesday. Today, I am providing one more example in a slightly different format. Recall that the invite requested that participants set out to accomplish something, make a plan and report on that “goal”, the plan, and the outcome.

So, let’s say I have discovered that I write too much in the passive voice. Based on that information, I would like to overcome the passivity in my writing voice, therefore my goal would be to learn how to write more assertively (less passively). In order to accomplish that goal, I may need to read up on the topic and learn exactly what it means to write passively. Then I would need to examine articles that I have written. And then I would need to practice writing more assertively. After all of that is done, I may have somebody (or something) analyze a brand new article or two to determine if I have achieved my desire.

After having executed on that plan, I will write about the experience including what the initial goal and plan were and also on what worked or didn’t work while trying to reach that goal. To summarize, here is an outline of that example:

What I will Accomplish

I will learn how to write more assertively (or just Write more assertively)

How Will I do that

Research what it means to write passively

Research what it means to write assertively

Evaluate “assertively” written articles

Take Notes on how to write assertively

Evaluate my articles

Practice writing assertively

Write a new article and have it reviewed to judge the voice whether it seems too passive or not

Report on The Successes and Failures

Write whether or not each step succeeded or failed.

Write if a step was unnecessary

Write about the experience and your thoughts about the experience.

Did you achieve or fail overall?

What is T-SQL Tuesday?

T-SQL Tuesday is a monthly blog party hosted by a different blogger each month. This blog party was started by Adam Machanic (blog|twitter). You can take part by posting your own participating post that fits the topic of the month and follows the requirements below. Additionally, if you are interested in hosting a future T-SQL Tuesday, contact Adam Machanic on his blog.

How to Participate

• Your post must be published between 00:00 GMT Tuesday, Août 9e, 2016, and 00:00 GMT Wednesday Août 10e, 2016.
• Your post must contain the T-SQL Tuesday logo from above and the image should link back to this blog post.
• Trackbacks should work. But, please do add a link to your post in the comments section below so everyone can see your work.
• Tweet about your post using the hash tag #TSQL2sDay.

A week ago, I saw a notification about the release of dbForge SQL Complete 5.5. As a long-time user of this product, I have updated to the new version. A week later I decided to share my impressions of the new version of SQL Complete.

The first thing that immediately caught my attention is the integration with SSMS 2016, which has recently become my main working tool. The new version of SSMS now includes a huge number of innovations, including Live Query Statistics that won my heart…

In addition to the SSMS 2016 integration, the new syntax of SQL Server 2016 was supported in SQL Complete. Previously, when writing articles on the new syntax, I had to look in MSDN sometimes, and now there is no necessity:

Previously, when typing GROUP or ORDER, BY was added automatically, and it seemed like a small revolution. In the new version, I liked the further development of this idea – to prompt more syntactic structures:

It is worth noting that phrase suggestion was implemented for DDL constructions:

Note also an improved suggestion inside the CROSS/OUTER APPLY constructions. I do not know for others, but these constructions are very useful for me. With their help, you can do UNPIVOT as shown in this picture:

As well as in individual cases influence the execution plan, forcing the optimizer to choose Index Seek when accessing data.

The context menu now includes the new command “Execute to cursor”, which came in handy a couple of times in practice:

What else can I say? Two new formatting profiles were added:

Now you can save time significantly without having to set up the formatting style for each construction from scratch. Here is an example of a badly formatted query:

select c.customername, o.orderdate,
ol.quantity
from sales.customers c join sales.orders o
on c.customerid = o.customerid join sales.orderlines
ol on o.orderid = ol.orderid
where c.isoncredithold = 0 and ol.unitprice > 0 order by o.orderdate desc,
ol.quantity

If you choose the first profile, after formatting you will get the following beauty:

SELECT c.CustomerName
     , o.OrderDate
     , ol.Quantity
FROM Sales.Customers c
    JOIN Sales.Orders o ON c.CustomerID = o.CustomerID
    JOIN Sales.OrderLines ol ON o.OrderID = ol.OrderID
WHERE c.IsOnCreditHold = 0
    AND ol.UnitPrice > 0
ORDER BY o.OrderDate DESC
       , ol.Quantity

If formatted with the second one, the result will be the following:

SELECT c.CustomerName,
       o.OrderDate,
       ol.Quantity
FROM Sales.Customers c
JOIN Sales.Orders o ON c.CustomerID = o.CustomerID
JOIN Sales.OrderLines ol ON o.OrderID = ol.OrderID
WHERE c.IsOnCreditHold = 0
    AND ol.UnitPrice > 0
ORDER BY o.OrderDate DESC,
         ol.Quantity

A small thing, but it makes a difference.

The rest of the improvements I saw “under the bonnet” – the speed when working with large scripts has increased (more than 1MB). This is especially true for those who often need to edit synchronization scripts of a schema or data.

Anyone who is interested in trying out the new version of SQL Complete can download it here.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL Complete – Smart Code Completion and SQL Formatting

One of the server settings that I always enable when configuring a new instance of SQL is database instant file initialisation

In case you don’t know what this is, it is a setting that can be granted to the service account that the SQL database engine runs under that prevents SQL Server from “zeroing out” new space when a data file is created or expanded.

This improves the performance of CREATE/ALTER DATABASE statements, RESTORE statements and AUTOGROWTH operations. A full detailed article to how and why can be found here.

It’s cool to see that you can now enable this when installing SQL Server: –

I can’t think of any downside to having this enabled (off the top of my head, there’s probably one or two) and it’s good to see that Microsoft know that most people enable it so adding it as an option in the installer is great imho.

Overview

SQL Server is the most popular relational database management system, which is used by many organizations to maintain their data. Every user expects that data should be stored in a consistent and integrated form. SQL Server is equipped with an in-built DBCC command, which allow users to check the integrity of the database. One of the command, DBCC CHECKDB helps to check integrity and allocation of specific objects in a database. This command should be executed on a regular basis to check if there is any corruption issue in the database files. It will analyze the entire database and provides the report regarding consistency of files, integrity of indexes, catalog consistency. So, it can be considered as an alert to detect the corruption in the database. However, the performance related issues associated with this command are a major area of concern for most of the SQL users. The following section aims to discuss on how to Minimize Performance Impact of SQL Server DBCC CHECKDB.

Impact of DBCC CHECKDB on SQL Server Performance

Although DBCC CHECKDB command is quite useful command to check the consistency but a number of issues are associated with this command, which hampers the performance of SQL Server. The top three performance related issues faced due to DBCC CHECKDB include:

1. System Process Delay:

When DBCC CHECKDB command is issued, it performs a thorough page-by-page analysis of the database files. Therefore, it acquires many resources at a time that slow down other system processes.

2. Slow Backup Process:

Since backup is a bit-to-bit copy of the existing database, it consumes much of the allocated resources as well as I/O devices. The backup file has to maintain every header, so it consumes lots of time.

3. Slow Recovery Process:

During the database recovery process, a hidden screen shot is created by DBCC. However, the consistency check of the screen shot is much time taking and slows down the recovery process.

Ways to Minimize Performance Impact of SQL Server DBCC CHECKDB

DBCC CHECKDB command plays a crucial role in the corruption issues, so it cannot be avoided due to its impact on performance. However, users can opt for measures to reduce the impact on performance of SQL Server. Some of the practices that can be followed while using DBCC CHECKDB are:

1. TempDb Maintenance:

DBCC makes a continuous use of TempDB database. Hence, it is advisable to store the TempDB on its own spindles so that any of the I/O spooling process does not get disturbed due to database access.

2. Database Mirroring:

Users can create a mirror image of the database using Database Mirroring setup. This will allow users to run DBCC CHECKDB on that version of database. It will keep the performance of primary database intact.

3. Avoid Information Messages:

Make sure that you use use DBCC CHECKDB with WITH NO_INFOMSGS to suppress all the irrelevant outputs generated during DBCC processing. It will somehow improve the performance rate of command processing.

4. Use Trace Flags:

In order to improve the performance, DBCC CHECKDB can be used with the following trace flags:

1. Trace Flag 2549: This trace flag optimizes the performance of SQL Server by treating each database file as it resides on a different disk. Thus, it makes a faster scan of the database files.
2. Trace Flag 2562: This flag treats the entire database as a batch and makes the use of better page reading algorithms. It will lead to a faster database processing.

5. Reduce CPU Impact:

DBCC CHECKDB command supports multithreading by default. In order to limit its CPU usage, users can reduce the parallelism in following ways:

• Trace Flag 2528 can be used to turn off parallelism during processing of DBCC CHECKDB.
• User can use SQL Server in-built feature “Maximum Degree of Parallelism” to specify the limit of CPU allocation.

6. Use Optimized Options

On a regular basis, DBCC CHECKDB can be used with other attributes, which can optimize the DBCC processing. Various options such as PHYSICAL_ONLY combined with CHECKSUM allows user to evaluate only physical record, thus helps in saving time and resource allocation.

Conclusion

Database corruption is a major hack, which damages user’s relevant data. It is always advised to perform a regular check to detect any corruption related issue. DBCC CHECKDB command is a significant utility to acknowledge the corruption in database. Once a corruption issue is detected in the files, users can take immediate steps to perform database recovery. However, it is always advisable to maintain an updated backup, which can be used in case of database failure or database corruption.

(This post and the link may be NSFW, be warned.)

I can’t believe that I have to write this.  First, before I continue look at the title of this blog post.  If you think I’m wrong, you really should keep reading.  If you agree with the title of this blog post then you probably don’t need to read it, but feel free to send it to someone who doesn’t.

If you’ve ever sent a picture of your *** to a person on the Internet, don’t.  She or he doesn’t want to see it.  I can pretty much guarantee that she doesn’t want to see it.  And from what my female friends are telling me it’s getting worse not better.  I can call these woman my friends because I go out for lunch and dinner with them, and have coffee with them, and have drinks with them, and yet I’ve never felt the need to send them a picture of my ***.

And while they’ve never thanked me for not sending them a picture of my ***, they don’t have to because NOT sending pictures of your *** to people is normal.  It’s you freaks that are sending out pictures of your *** to random women that are the odd ones.

I get that you think you’re being funny, or sexy and that you think that the women that you send the picture to will like it, and will want to have sex with you.  But she won’t.

Here’s a story of what one women did when someone sent her a *** pic.  And I think every woman out there should do the exact same thing.

If you’ve made it this far and you still think that sending pictures of your *** to women is acceptable, all I can tell is that you are wrong.  And I really hope that you enjoy women sending you back all the *** pics that she’s gotten.  And know that she’ll be adding yours to the collection of *** pics being sent out to guys that send them to her.

And if you think that her sending you *** pics isn’t fare because you don’t want to see them, now you know how she feels.

Denny

The post No, she really doesn’t want a picture of your ***! appeared first on SQL Server with Mr. Denny.

A DBA walked into a bar, saw a couple of tables, and asked “Can I JOIN you?”

There are several different kinds of JOINs. OUTER JOINs are one of the more complex options. With an INNER JOIN only those rows that have a match on both sides of the JOIN are returned. With an OUTER JOIN unmatched rows are returned also.

That’s where the LEFT, RIGHT and FULL comes in. They determine which unmatched rows are returned. Per BOL:

INNER

Specifies all matching pairs of rows are returned. Discards unmatched rows from both tables. When no join type is specified, this is the default.

 
FULL [ OUTER ]

Specifies that a row from either the left or right table that does not meet the join condition is included in the result set, and output columns that correspond to the other table are set to NULL. This is in addition to all rows typically returned by the INNER JOIN.

 
LEFT [ OUTER ]

Specifies that all rows from the left table not meeting the join condition are included in the result set, and output columns from the other table are set to NULL in addition to all rows returned by the inner join.

 
RIGHT [OUTER]

Specifies all rows from the right table not meeting the join condition are included in the result set, and output columns that correspond to the other table are set to NULL, in addition to all rows returned by the inner join.

Well, I hope that helps a bit. Of course most queries look like this:

SELECT *
FROM sys.tables t
LEFT OUTER JOIN sys.columns c
	ON t.object_id = c.object_id

So what’s the left and what’s the right? Really it’s up and down correct?

Nope. The missing piece of knowledge here is that T-SQL ignores white space (more than required single spaces, tabs, carriage returns, etc). So when the optimizer looks at the query it sees this:

SELECT * FROM sys.tables t LEFT OUTER JOIN sys.columns c ON t.object_id = c.object_id

Now we have a left and a right. And now the LEFT and RIGHT makes a whole lot more sense.

This simple blog will cover the bulk of information you’ll need to make a fully informed decision on which Recovery Model to choose for your SQL Server databases. I know, that was horrible, but I couldn’t resist. Recovery Models are normally considered one of the more basic things to know about SQL Server, but refreshers are never a bad idea. Furthermore, if you are like me, you haven’t actually dealt with the Bulk Logged model much.

Simple Recovery Model

The simple model constantly reclaims log space in order to keep storage requirements minimal. Transactions are still written to the log; however, once a transaction completes and the data is written to the data file, the transaction log is truncated. This keeps the log small and allows the space to be reused.

While this model takes the least amount of planning and work to maintain, it also provides the least amount of protection. You cannot perform transaction log backups, so any changes since your last backup will be lost in the event of a disaster. There is no possibility to recover to a specific point in time either, and high availability solutions cannot be implemented. This model is best used for unimportant or test data. Data that rarely changes or would have no impact if data loss occurs between backups are also good choices for Simple recovery.

Supported Backups:

• Full
• Differential

Unavailable:

• Log shipping
• Database Mirroring
• Always On Availability Groups
• Point in time Restore

Bulk Logged Recovery Model

This model performs minimal logging of bulk operations in order to reduce the size of the transaction log. This means that the model does support the use of transaction log backups. In fact, it’s essentially identical to the Full Recovery model except for omitting bulk operations from the log. Even so, you shouldn’t use this model long term, just go full.

Bulk operation examples:

• Bulk Imports (BCP, Bulk Insert)
• CREATE/ALTER INDEX
• SELECT INTO
• WRITETEXT & UPDATETEXT

Since Point in Time recovery is not supported in this model, data can be lost, but only if the transaction log is damaged or bulk logged operations occur (since those aren’t logged). This recovery model is best used during index maintenance or when inserting massive amounts of data. Your transaction log won’t blow up in size, and you’ll be able to flip back to Full Recovery easily after your maintenance is complete (more on that later).

Supported Backups:

• Full
• Differential
• Transaction Log

Full Recovery Model

With the Full Recovery model, you can prevent data being lost due to damage and you finally have the opportunity to restore to a specific point in time. All transactions are stored in the log until a log backup occurs or the log is truncated. If you fail to schedule regular log backups, your transaction log will grow till storage becomes a major problem. After a transaction completes, the data from the log is then written into the data file.

If the tail of the log is damaged, you can still have some data loss up to the last transaction log backup. This is the most appropriate recovery model to use for production data as its the only way to provide point in time recovery.

Supported Backups:

• Full
• Differential
• Transaction Log

Changing Recovery Models

The command to flip recovery models is very easy, just issue an alter command on the desired database and set it to the desired model.

USE [Master];
ALTER DATABASE TestDB SET RECOVERY FULL;

Flipping between recovery models requires a few considerations, which I’ve summed below.

[Notes from Pinal]: Performance tuning is a complex subject, and not everyone knows where to start and how to handle this complex subject. There are so many different areas for performance tuning for example – tuning TempDB, the bottleneck at SAN, disk latency, etc. If a novice user start on the subject of performance tuning they will easily get lost as they do not know where to start. This is where I requested help from Edurado and he suggested that we should take help from DMVs to detect performance problems in SQL Server.

Linchpin People are database coaches and wellness experts for a data driven world. In this 131st episode of the Notes from the Fields series database expert Eduardo Castro (partner at Linchpin People) shares very interesting conversation related to we can get started with R Services and Forecasting.

This blog contains some T-SQL Scripts and DMV that can be used to detect performance problems in SQL Server and will help to understand the origin of the problem.

When you have to optimize a SQL Server should review the following infrastructure components and programming:

• Bottlenecks in tempdb. Each instance of SQL Server has a unique tempdb database, which may involve performance problems, since we can a SQL Server instance with contention in tempdb or something more serious as lack of space. A poorly constructed TSQL using excessive resources can affect the performance of tempdb. This may cause other applications or databases that use tempdb to be affected in their performance.

• A query that runs very slow. The performance of existing queries may change because of various factors such as statistics have not been updated or indexes that have not been rebuilt or worse missing indexes causing table scans.

• Changes in the system disk or SAN, a change in the type of disc that is being used can affect the performance.

• When queries are optimized, often slow applications are due to blockages that are being generated due to poor design of the application or because levels has elected not suitable insulation.

Based on the previous possible causes of  bad performance I will show some  DMV that can be used to detect performance problems with SQL Server.

If we want to see if we have problems of CPU, we can use the DMV sys.dm_os_schedulers. This DMV gives you information about schedulers in SQL Server, usually in an environment without performance issues CPU values of this DMV tend to be zero, therefore, values that are greater than zero means that there are tasks that have to wait for be executed, if these values are too high, then we are in the presence of a problem of CPU capacity.

SELECT scheduler_id, current_tasks_count, runnable_tasks_count, 
current_workers_count, active_workers_count, context_switches_count, 
work_queue_count, pending_disk_io_count 
FROM sys.dm_os_schedulers 
WHERE scheduler_id <255

We can interpret the values as follows:

– Runnable_tasks_count should be zero in most cases

– Current_workers_count indicates the number of workers that are associated with this scheduler.

– Work_queue_count number of tasks that are waiting to be assigned to a worker

– Pending_disk_io_count Indicates IO which is waiting to be completed.

Another component that can be checked to optimize SQL Server is the Buffer Pool, which is the component that stores and manages the data cache in SQL Server. Detailed information on the Buffer Pool can be obtained by DMV sys.dm_os_buffer_descriptors, this DMV returns information on all pages that are being used by the Buffer Pool, based on these results we can see the distribution of the Buffer Pool according to usage, it returns the pagetype which can be a data Page, Index Page or TEXT_MIX_PAGE (more information on pages types available).

If we execute the following statement

SELECT * 
FROM sys.dm_os_buffer_descriptors

Distribution information is obtained from the Buffer Pool. In this result, we see all pages are being used by the buffer pool, remember that each page of SQL Server is 8KB, in this case each data page has a descriptor cache buffer. Buffer descriptors uniquely represent each page that is currently being used in the cache of an SQL Server instance.

If we want a summary view of this information we can run the following T-SQL.

SELECT count (database_id) * 8/1024 AS 'Cache Used (MB)' 
, CASE database_id 
	WHEN 32767 THEN database_id 'BD Resources' 
	ELSE db_name (database_id) 
	END AS 'DatabaseName' 
FROM sys.dm_os_buffer_descriptors 
GROUP BY db_name (database_id), database_id

This T-SQL gives us the amount of cache used by each database from the current instance of SQL Server.

If we want to see if we have problems of physical memory on the server can view information on the buffer manager.

SELECT * 
FROM SYS.SYSPERFINFO 
WHERE OBJECT_NAME = 'SQLServer: Buffer Manager'

From the information above, we can verify the values of Target Stolen Pages and Pages. The stolen Pages counter indicates the number of pages that were “stolen” from the cache to meet the demand for physical memory. In this case the number of Stolen Pages should remain stable over the course of time, if it is not, then we have a problem of the amount of available physical memory.

The SQL Server counter: Buffer Manager – pages Database, indicates the number of pages from the cache, usually this value is stable over time, if abrupt changes occur indicates that is becoming swap cache, so both could be an indication that we need to increase physical server memory.

The counter SQL Server: Buffer Manager – Buffer cache hit ratio indicates the percentage of pages that have been found in memory, therefore, the higher the better value.

The SQL Server counter: Page life expectancy represents the average number of seconds that a machine remains in the cache, for OLTP systems one Page life expectancy, average of 300 is five minutes, a lower value that may represent a memory problem.

Since SQL Server version 2008 it includes system_health which is a XEvent that helps us make troubleshooting performance SQL Server engine. If we run the following command:

SELECT * 
FROM sys.dm_xe_sessions

If we want to see the results of this XEvent run the following T-SQL

SELECT CAST(xe_targets.target_data as xml) 
FROM sys.dm_xe_session_targets xe_targets  
JOIN sys.dm_xe_sessions xe_sessions  
ON (xe_sessions.address = xe_targets.event_session_address)  
WHERE xe_sessions.name = 'system_health'  

This T-SQL will give us the results in XML format with the corresponding information.

If we open that XML will see something like the following

If the SQL Server engine is having problems, then we can review the XML that contains information about the memory, latches, waits, and more detail information about SQL Server.

If you want me to take a look at your server and its settings, or if your server is facing any issue we can Fix Your SQL Server.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – DMVs to Detect Performance Problems in SQL Server – Notes from the Field #135