I’m happy, honoured and humbled to be able to say that I will be presenting pre-cons and both SQL Saturday 496 in Exeter and SQL Saturday 502 in Edinburgh.

SQL Saturday 496 Exeter is on Friday 11th March and Saturday 12th March

The event home page is here and signup to my class is :

http://sqlsatexeter.azurewebsites.net/?page_id=5561

SQL Saturday 502 Edinburgh is on Friday 10th and Saturday 11th Jun

The event home page is here and signup to my class is :

https://www.eventbrite.com/e/sql-server-2016-masterclass-query-processing-and-internals-tickets-20699726439

Both will follow the same format and general material so no need to worry that you’ll miss out on something if you can only attend one and not the other, though i would suspect that distance would be a better differentiator  We will be looking at the Query Optimizer and delving into its internals to find how it builds a plan,  what its limits and limitations are and understanding how to recognize that we have a ‘Bad Plan’.

Hope that you can make one or the other.

In the past, I have written about attaching data files to a SQL Server instance when log files are missing by the use of the FOR ATTACH_REBUILD_LOG clause of the CREATE DATABASE statement. Recently, I was referring the CREATE DATABASE documentation on MSDN. It’s a comprehensive document and and the following line for read-only databases caught my attention.

For a read-only database, the log cannot be rebuilt because the primary file cannot be updated. Therefore, when you attach a read-only database with a log that is unavailable, you must provide the log files, or the files in the FOR ATTACH clause.

This statement was contrary to what I had observed before. Hence, I decided to re-validate the findings via a demo.

The first step, of course is to create a database.

USE [master];
GO
CREATE DATABASE ReadOnlyDBForAttach
ON PRIMARY (NAME = ReadOnlyDBForAttach_Data,
            FILENAME='C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_Data.mdf'),
   FILEGROUP RODefault (NAME = ReadOnlyDBForAttach_RODefault,
              FILENAME='C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_RODefault.mdf')
LOG ON (NAME = ReadOnlyDBForAttach_Log,
        FILENAME='C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_Log.ldf');
GO

As can be seen from the script referenced above, the database we created as 2 data file groups, one is the default PRIMARY filegroup. Because we cannot make the PRIMARY filegroup READ_ONLY, we will be making the other filegroup (RODefault) READ_ONLY. To add complexity, we will also make it the default filegroup, i.e. any new objects created without specifying a filegroup, it will be created in the filegroup marked as default.

Finally, we will also mark the database as READ_ONLY.

USE [master];
GO
ALTER DATABASE ReadOnlyDBForAttach
MODIFY FILEGROUP [RODefault] DEFAULT;
GO

ALTER DATABASE ReadOnlyDBForAttach
MODIFY FILEGROUP [RODefault] READ_ONLY;
GO

ALTER DATABASE ReadOnlyDBForAttach SET READ_ONLY;
GO

Now, let us check out the database and filegroup properties.

USE ReadOnlyDBForAttach;
GO
SELECT 'Database Properties',
       sd.[name],
       sd.is_read_only,
       sd.is_cleanly_shutdown
FROM sys.databases AS sd
WHERE sd.[name] = 'ReadOnlyDBForAttach';

SELECT 'File properties',
       sdf.file_id,
       sdf.type,
       sdf.data_space_id,
       sdf.type_desc,
       sdf.name,
       sdf.is_read_only,
       sdf.is_media_read_only
FROM sys.database_files AS sdf;
GO

Read Only database and database file properties

Now, let us detach the database, and delete the log file. (NOTE: We are removing the log file for the purposes of this demo only. Please do not do this in your QA or production environments).

USE [master];
GO
EXEC sp_detach_db @dbname = 'ReadOnlyDBForAttach';
GO

Read Only Database – Log File Removed

Finally, let us attach the database back to the SQL Server instance using the CREATE DATABASE…FOR ATTACH_REBUILD_LOG clause.

USE [master]
GO
CREATE DATABASE [ReadOnlyDBForAttach]
ON  PRIMARY ( NAME = N'ReadOnlyDBForAttach_Data',
                FILENAME = N'C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_Data.mdf')
FOR ATTACH_REBUILD_LOG
GO

File activation failure. The physical file name "C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_Log.ldf" may be incorrect.New log file 'C:\SQLData\MSSQL12.SQL2014\MSSQL\DATA\ReadOnlyDBForAttach_log.ldf' was created.

As can be seen from the message above, the log file was successfully created and the database was successfully attached to the SQL Server instance.

Let us cross-check the database and file properties again:

USE ReadOnlyDBForAttach;
GO
SELECT 'Database Properties',
       sd.[name],
       sd.is_read_only,
       sd.is_cleanly_shutdown
FROM sys.databases AS sd
WHERE sd.[name] = 'ReadOnlyDBForAttach';

SELECT 'File properties',
       sdf.file_id,
       sdf.type,
       sdf.data_space_id,
       sdf.type_desc,
       sdf.name,
       sdf.is_read_only,
       sdf.is_media_read_only
FROM sys.database_files AS sdf;
GO

Read Only Database and Data File Properties After Attach

Conclusion

• Contrary to the MSDN remark, a read-only database can be successfully attached to a SQL Server instance even when the log file does not exist by the use of CREATE DATABASE…FOR ATTACH_REBUILD_LOG
• A read-only database becomes a read/write database if it has been attached to the SQL Server using FOR ATTACH_REBUILD_LOG and the log file was rebuilt

My findings above do not agree with the MSDN remark. At the moment, I am inclined to believe that this is a bug in the documentation as I have found a couple of years ago as well (see references). Please do share your views on the same in the post comments.

References/Further Reading

• SQL Server Myth: Log files are removed when a database is made READ_ONLY [Link]
• Creating a database without the log backup file – Error Msg. 5120 [Link]
• CREATE DATABASE…ATTACH_REBUILD_LOG resets database recovery mode to SIMPLE [Link]
• Setting database to READ_ONLY does not change the file-group properties [Link]
• sp_attach_db v/s CREATE DATABASE FOR ATTACH – Number of files limitation; Msg. 8144 (Procedure has too many arguments specified) [Link]
• CREATE DATABASE [MSDN Documentation]

Until we meet next time,
Be courteous. Drive responsibly.

To the page

It’s not one of those things you have to do frequently but every now and again you need to know what page a particular row is on. It’s not terribly difficult. There is a virtual column called %%physloc%%, but unfortunately it’s not terribly useful on it’s own.

USE AdventureWorks2014
GO
SELECT %%physloc%%, * 
FROM Purchasing.ShipMethod
GO

But if we use one of the two system functions specifically designed to work with it things improve dramatically.

USE AdventureWorks2014
GO
SELECT sys.fn_PhysLocFormatter(%%physloc%%) PhysLocFormatter, 
	PhysLocCracker.*, Purchasing.ShipMethod.* 
FROM Purchasing.ShipMethod
CROSS APPLY sys.fn_PhysLocCracker(%%physloc%%) PhysLocCracker
GO

So now we have the page each row is located on. Of course this may not be the only page that a row has data in, but it is the page with the in row data.

And back again

More commonly we have a page location and want to know the table associated with it. I was reading a post by Matan Yungman (b/t) recently. When Should You Use Identity as a Clustered Index Key? And one of the things he pointed out was that sometimes you’re looking at sys.dm_os_waiting_tasks, or sys.dm_exec_requests, etc and you have a page number for a resource being used. So what’s an easy way to get back to the object? Well you can use DBCC PAGE to look at the information in the page but honestly I find that a bit cumbersome. I find it quite a bit easier to use a system view that Jason Strate (b/t) posted about a while ago. sys.dm_db_database_page_allocations You can plug in the database id and include the page and file ids in the WHERE clause.

I should probably stop here and explain how the page numbers are usually formatted.

Sometimes you will see (1:820:3)

• Position 1: 1: This is the file id.
• Position 2: 820: This the page id.
• Position 3: 3: This is the slot id. The location within the page that the row exists.

 
Other times you see 6:1:820

• Position 1: 6: This is the database id.
• Position 2: 1: This the file id.
• Position 3: 820: This is the page id.

 
From this information you can easily get the table name and schema.

SELECT object_schema_name(object_id, database_id) Object_Schema, 
	object_name(object_id,database_id) Object_Name
FROM sys.dm_db_database_page_allocations(6,null,null,null,null)
WHERE allocated_page_file_id = 1
  AND allocated_page_page_id = 820

I want to point out that everything in this post is undocumented so you use it at your own risk.

If you're using OPENQUERY and get this error:

OLE DB provider "SQLNCLI10" for linked server "MYREMOTESERVER" returned message "Deferred prepare could not be completed.".
Msg 916, Level 14, State 1, Line 1
The server principal "linkedServer" is not able to access the database "MyDatabase" under the current security context.

Check to be sure the SID of the login is the same as the SID of the database's user:

-- When copying a database from another server, you will need to run this -- if the instance already has a login with the same name (because the SIDs -- will be different because they came from different instances). -- SELECT DATABASE !!!  
use MyDatabase

-- Use this to get instance login sid.  
select SUSER_SID('linkedServer')

-- Display a list of users in current database and their User SID. 
exec sp_change_users_login @Action='report'

-- Set database user SID to corresponding instance login SID.  
alter user linkedServer with login = linkedServer

Did you use to use Yahoo! Pipes to manipulate RSS Feeds? Personal I did and I REALLY missed it when it was turned off by Yahoo. So like any good IT person I sat down (eventually) and did something about it. What I ended up building turned into what I’ve called “Feed Monsters“.  Feed Monsters is a web based service that lets you change values within RSS feeds, resort the values within the RSS feeds and filter the RSS feeds.

After you get signed up you create your monster and give it an RSS feed to chew on.  Then you add parts to your monster and those parts each make a different change to the RSS feed that you’ve given it.  As the service is being launched it supports 10 different parts that you can add to your monster.

Data Change Parts

• Add Category from Publish Date
• Add Prefix to Title
• Add Suffix To Title
• Clip Content
• Post URL
• Publish Date
• Replace field {x} with field {y}
• Title

Filter Parts

• Top {n} Posts

Sorting Parts

• Random Resorting

There’s plenty of more parts which we’ll be adding to the system in order to make RSS feeds even more changeable in the near future.

Registration for the service is free, and lets you try the service out for 10 days without having to pay anything. After that it’s only $20 a year (paid for paypal) to continue using the service.  Hopefully you find the service as valuable to use as I do.  Check out the Feed Monsters website for the full details about the service.

Denny

The post Announcing Feed Monsters… appeared first on SQL Server with Mr. Denny.

Normal DBAs will sum up shrinking a database data file in one word: NO!

I’ll talk to you about this subject because I’m not normal.  Still, my goal is that you’ll go from “why can’t I shrink my files” to “I don’t want to shrink my files.”

Truth be told, even when I HAVE to shrink files, I don’t want to.

Basically, you needed the space at one point in time, so you’ll probably need it again.  Unless you changed a process or something extremely rare happened, you’ll use the space again and there’s no reason to take on the issues involved with shrinking just to have a file grow again.  While this advice is mostly the same for log files, it’s different enough where I’m making a separate post for shrinking database log files

When should you shrink data files

First, a process changed where you’ll never need that amount of space again, such as the database use to keep 10 years of history but now an automated process keeps it pruned to 6 months.  This change also left you with a very significant amount of free space that you need for another purpose.

Really, that’s about it.

What happens when you shrink data files

The most common way to shrink a file is to have it reorganize pages before releasing free space, so I’ll cover that.  It basically has two steps:

1. Fragment the file as much as possible (killing performance)
2. Truncate the end of the file

By “reorganize” it means take the tail end of the file after the size you said you’d like the file to be shrunk to, find every populated page of data after that point, then move those pages to the first available spot you find in the file.  When the end of the file is empty, truncate it.

It’s an expensive process that leaves a big mess behind, but there are times you’ll need to go through with it (kicking and screaming).  If you do, schedule an index maintenance job immediately after this to undo most of the damage you just did and make sure the server still loves you.

Think about this, you’re considering kicking off a process where you have to tell the server you’re sorry afterwards.  You better have a good reason for this.  A 100 GB database with only 2 GB used may be a good reason to shrink.  A 100 GB database with 75 GB used is normal and healthy.

What about the Auto-Shrink option

No.

Just no.

It’s not a feature, it’s a threat.

If you have a certain amount of free space it will automatically shrink your database causing all the problems I just discussed, but it will probably kick in during your busiest time of day.  Then it won’t follow that up with index maintenance, so it just left you hopping all over your disk to find data.  The best part is that it didn’t check to see if you would need that space again, so it’ll probably grow tomorrow to get that space back.

The Connect Item on this is marked as fixed, stating that they’ll consider removing it in future versions.  This wasn’t a joke opened by some wannabe DBA.  This was serious and opened by someone who went on to become the president of the Professional Association for SQL Server.

To be fair, we all word our feelings on this differently.  I say no, Tom LaRock says never, Paul Randal says Turn It Off!, and Brent Ozar is colorful enough to come up with the term Hamster Wheel of Death.

As of the time of this writing, no one has used the term “Magical Unicorn” to describe this feature.

Sum it up

1. Don’t shrink data files.
2. If you didn’t listen to #1, why?
3. Follow it up with index maintenance.
4. Every time you try to automate this process a unicorn dies.

I recently came across Tim Ford’s (B|T) challenge to the technical blogging community. The challenge is to write one introductory level blog per month for the community.

A simple statement with a profound substance to it. How many times do you and I take for granted the knowledge we have and share both privately and socially on a daily basis?

More recently, I was approached with a question along these lines that I interpreted to be simplistic in form. The data professional however was sincere and it made me realize that there are still a lot of gaps to bridge…..and more so, who am I to judge what is simplistic or not?

It’s time to set ego’s aside and the game of “can you top this” with our knowledge and continue to bridge these types of gaps amongst members in our community and beyond.

No question is a “dumb” question for it is how we learn and grow as professionals. To the new community member or data professional keep asking those questions and being ambitious.

A huge thank you to Tim for bringing this challenge more to the forefront; it is something I, myself, have taken for granted. It was an eye-opening experience to know where I came from, where I am now, and where I want to go. May we never stop learning and sharing.

Onward….let’s roll!

Let’s say you have a port conflict and need to change the port on your Availability Group endpoint.  How can we accomplish this?

The first step is to change the port on the endpoint itself.  Note that as soon as you do this the replica on which you are running it will no longer be able to replicate data until you have updated the URL as well.  Here is the code to change the port, just make sure you know what the IP is unless you are changing that too.  If you are, then read this post on changing the endpoint IP address.

ALTER ENDPOINT [MyEndpoint]

STATE = STARTED

AS TCP (LISTENER_PORT = 7777, LISTENER_IP = (10.x.x.x))

FOR DATA_MIRRORING (ROLE = ALL, AUTHENTICATION = Windows Negotiate, ENCRYPTION = REQUIRED ALGORITHM AES)

GO

The second step is to change the replica URL.  This part is pretty straight forward since you already know the port from above.  What I want to encourage you to do is use the full URL or FQDN for the server name.

ALTER AVAILABILITY GROUP [MyAG]

MODIFY REPLICA ON 'MyNode1' WITH (ENDPOINT_URL = 'TCP://MyNode1.mydomain.com:7777')

GO

That’s all there is to it!

Yet another new M function for you this week: the Table.Schema() function, which returns information about the columns in a table. There’s some fairly detailed documentation about what it returns here; a simple demo is always a lot more helpful though, I think.

If you connect to the Adventure Works DW database in SQL Server and import the DimDate table, you’ll get an M query that returns the contents of that table (along with some extra columns that describe the relationships between that table and the others in the database):

let
    Source = Sql.Database("localhost", "adventure works dw"),
    dbo_DimDate = Source{[Schema="dbo",Item="DimDate"]}[Data]
in
    dbo_DimDate

If you add an extra step to this query that calls the Table.Schema() function on the table returned by the dbo_DimDate step, like so:

let
    Source = Sql.Database("localhost", "adventure works dw"),
    dbo_DimDate = Source{[Schema="dbo",Item="DimDate"]}[Data],
    GetSchema = Table.Schema(dbo_DimDate)
in
    GetSchema 

…you get a table with one row for each column in the table returned by dbo_DimDate, and a lot of columns that give you information on each column such as its position in the table, its M data type, its data type in the original data source and so on:

I can think of a lot of uses for this. Documenting a database is an obvious one; it would also serve as a much richer source of data when checking for changes in the structure of a data source, as I described here. Also, given how easy it is to tell whether two tables contain the same data in M, you could use this function to compare whether two tables have the same columns like so:

let
    Source = Sql.Database("localhost", "adventure works dw"),
    dbo_DimDate = Source{[Schema="dbo",Item="DimDate"]}[Data],
    SomeOtherTable = 
     Source{[Schema="dbo",Item="SomeOtherTable"]}[Data],
    TablesEquivalent = 
     (Table.Schema(dbo_DimDate)=Table.Schema(SomeOtherTable ))
in
    TablesEquivalent

If you want more detail when doing comparisons you can do that with a little bit more M code, but that’s probably a topic for another post..

The dmv dm_exec_query_stats doesn’t track stats for OPEN CURSOR statements. This is a problem because the OPEN statement is the one that “runs” your query and if you rely on these stats to monitor performance, then cursor performance is hidden from you.

Cursors have a bad reputation, probably well-deserved. When I see a cursor, I see someone trying to use SQL as a programming language. It’s not what SQL is good at and there’s often a better way.

The pragmatist in me doesn’t care too much. If a cursor is performing well and not causing too much trouble, then fixing will not be a priority. But my monitoring solution doesn’t show me how expensive those cursors are! I realize I have no idea what my cursors are doing or how expensive they are.

Cursor Statements

Developers use a number of SQL Statements when writing cursors: DECLARE, OPEN and FETCH. Performance-wise, the DECLARE CURSOR statement takes no time. The OPEN statement runs the query and puts the results in a temporary table. And the FETCH statement reads the next row from the table.

If a cursor’s query is untuned, it’s the OPEN statement that consumes the most resources.

Example

The OPEN statement is missing from sys.dm_exec_query_stats. I want to demonstrate that. Run the following on a dev box.

-- fresh start:DBCC FREEPROCCACHE
SETSTATISTICS IO ONSETNOCOUNTON;
GO
 
-- declare a cursor with an arbitrary query that reads a little bitDECLARE @ChecksumValue int;
 
print'declare cursor:'DECLARE FiveRows CURSORLOCAL FAST_FORWARD FORSELECTTOP5CHECKSUM(*)FROMINFORMATION_SCHEMA.COLUMNSORDERBYCHECKSUM(*) 
-- this statement actually runs the query that was just declaredprint'open cursor:'OPEN FiveRows
 
-- fetch the five rows one at a timeDECLARE @i INT=0;
WHILE( @i <5)BEGINprint'fetch cursor:'FETCHNEXTFROM FiveRows INTO @ChecksumValue
 
    SET @i +=1;
END 
CLOSE FiveRows
DEALLOCATE FiveRows;
 
GO
 
-- Now look at dm_exec_query_text to see what's in thereSELECT 
    qs.query_hashas QueryHash,
    qs.total_logical_reads+ total_logical_writes as TotalIO,
    qs.execution_countas Executions,
    SUBSTRING(
        st.[text],
        qs.statement_start_offset/2,
        (qs.statement_end_offset- qs.statement_start_offset)/2)as SQLText    
FROM sys.dm_exec_query_stats qs
OUTER APPLY sys.dm_exec_sql_text(qs.[sql_handle]) st
ORDERBY qs.total_logical_reads+ total_logical_writes DESCOPTION(RECOMPILE)

The results of that last query show that the OPEN statement is missing from dm_exec_query_stats:

And the messages tab shows that the OPEN statement did in fact read from tables.

declare cursor:
open cursor:
Table 'Worktable'. Scan count 0, logical reads 21, ...
Table 'syscolpars'. Scan count 1, logical reads 15, ...
Table 'sysschobjs'. Scan count 1, logical reads 38, ...
Table 'sysscalartypes'. Scan count 1, logical reads 2, ...
fetch cursor:
Table 'Worktable'. Scan count 0, logical reads 2, ...
fetch cursor:
Table 'Worktable'. Scan count 0, logical reads 2, ...
fetch cursor:
Table 'Worktable'. Scan count 0, logical reads 2, ...
fetch cursor:
Table 'Worktable'. Scan count 0, logical reads 2, ...
fetch cursor:
Table 'Worktable'. Scan count 0, logical reads 2, ...


Workarounds

If your cursors are defined inside a procedure, you can inspect dm_exec_procedure_stats. This is not an option when cursors are run as ad-hoc SQL (outside a procedure). Remember that you’ll only get the performance numbers for the entire execution of the procedure. This view doesn’t tell you which statements inside the procedures are expensive.

There’s good news if your monitoring solution is based on extended events or SQL Trace. You’ll be able to monitor cursors correctly.

If you plan to use Query Store, the new feature in SQL Server 2016, then you’ll be able to see statistics for the OPEN query. Query Store doesn’t store statistics for the DECLARE statement. But that’s acceptable because DECLARE statement don’t use any resources.

Summary

Use the following to keep everything straight.

	DECLARE CURSOR	OPEN CURSOR	FETCH CURSOR
dm_exec_query_stats			(queryhash = 0x0)
dm_exec_procedure_stats (if run as sproc)	(aggregated)
SQL Trace (e.g. Profiler)
SET STATISTICS IO ON
Show Plan	*
Query Store (2016)	*

* This is acceptable because we don’t care about the performance of DECLARE statements.

tl;dr; Re-start the instance in safe mode (-f startup parameter) and move tempdb. Then re-start without the parameter.

While at the 2015 Summit I walked into the last few minutes of Russ Thomas’s (b/t) session on Stress Inoculation: Maintaining Performance Under Pressure. As a side note I was under some heavy stress myself at that particular moment (I was waiting for my session to start) which explains why I was wandering around the hall aimlessly.

From what I saw it was a great session. The premis is that you won’t be nearly as stressed out during a disaster if you’ve practiced resolving that particular disaster. Sounds reasonable right? As part of his session he did these great interactive demos. He brought people up from the audience and had them try to solve a difficult problem while he harassed them (gently) in the background. Anyone who has dealt with a server crash (for example) will realize this is a pretty good way to demonstrate the type of stress you might be experiencing. At the moment I walked in one of the audience members was being challenged. He was being asked to re-start a SQL Instance where the drive that held tempdb was no longer available. As Russ put it this is a low occurance high liability issue. (It doesn’t happen very often but when it does it’s a big deal.)

So in case it ever happens to you (or me) here we go:

Starting a SQL Server without tempdb

The situation: Your server is down. The drive/directory where tempdb is supposed to be doesn’t exist. Who knows why. Maybe those evil SAN guys forgot to re-attach your storage during a DR situation. You may or may not realize it but SQL Server will not start without tempdb. Which is fine. Just move it to a location that exists right? Well, yes. That is an important step. So here is how we

Move tempdb

1. Run an alter database

   -- Pulled straight out of MSDB
   USE master;
   GO
   ALTER DATABASE tempdb 
   MODIFY FILE (NAME = tempdev, FILENAME = 'E:\SQLData\tempdb.mdf');
   GO
   ALTER DATABASE tempdb 
   MODIFY FILE (NAME = templog, FILENAME = 'F:\SQLLog\templog.ldf');
   GO

2. Stop and re-start SQL Server

 
Ok, so if you can tell me how to run an ALTER DATABASE when the instance is down I’ll be hugely impressed. Actually beyond impressed. Stunned even. We are going to have to move tempdb though. We will need it once we are back up again.

Step one is to bring the instance back up in some form or another. There are two ways (well that I can think of off the top of my head) to start up SQL Server. We can use the sqlservr.exe command to run from a command shell or use SSCM (SQL Server Configuration Manager).

Either way we need to start up SQL Server with minimal configuration (use parameter -f). One of the benefits of minimal configuration is that it doesn’t require tempdb. Be warned it also puts the instance in single user mode. So if you start the instance up and the Agent starts up with it you will need to shut it down before you can connect to the instance. Also as with any time you are in single user mode you only want to connect via SQLCMD or a query window in SSMS (no object explorer).

Note: You can also use trace flag 3608 which tells SQL Server not to start up or recover any database but Master.

Now that SQL Server is started up we can use the above code to move tempdb to an existing location and then re-start SQL Server without the startup flag or trace flag.

And after that nice long winded explanation here is the quick and easy demo.

USE master;
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = tempdev, FILENAME = 'c:\tempdb\tempdb.mdf');
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = templog, FILENAME = 'c:\tempdb\templog.ldf');
GO

Next I shut down that instance and deleted the directory. Now when I try to start the instance I get these errors in the event log.

CREATE FILE encountered operating system error 3(The system cannot find the path specified.) while attempting to open or create the physical file ‘c:\Program Files\Microsoft SQL Server\MSSQL11.SQLEXPRESS2012\MSSQL\DATA\tempdb\tempdb.mdf’.
FCB::Open failed: Could not open file c:\Program Files\Microsoft SQL Server\MSSQL11.SQLEXPRESS2012\MSSQL\DATA\tempdb\tempdb.mdf for file number 1. OS error: 3(The system cannot find the path specified.).
The SQL Server (SQLEXPRESS2012) service terminated with the following service-specific error:
The specified resource name cannot be found in the image file.

Not terribly surprising right? I do want to point out that this tells us exactly what the error is. There are any number of reasons an instance might not start, this error tells us exactly what we need to do.

Now in this particular case we can fix the problem by just re-creating the missing directory but let’s pretend that isn’t an option (the whole drive is missing for instance). So let’s re-start the instance using the -f flag. First I open SSCM and go to the properties of the instance.

Then on the Startup Parameters tab I add a parameter -f.

Hit the Apply button then re-start the instance.

Now we move tempdb back where it belongs. Just to make the whole process fun I got this nifty error when connecting to the instance:

I’m not sure why but I connected anyway. Once connected we run our fix script:

USE master;
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = tempdev, FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL11.SQLEXPRESS2012\MSSQL\DATA\tempdb.mdf');
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = templog, FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL11.SQLEXPRESS2012\MSSQL\DATA\templog.ldf');
GO

Now back to SSCM and the instance properties where we remove the -f flag.

Restart the instance again.

And we are back to normal.

Alright so maybe that wasn’t as quick as I said. It was easy though wasn’t it :)

Azure SQL Database is a relational database-as-a-service in the cloud.  It uses a special version of Microsoft SQL Server as its backend that is nearly identical to SQL Server (see Azure SQL Database Transact-SQL differences).  While there are many benefits to using SQL Database over SQL Server, in this blog post I’ll talk about the various types of high-availability and disaster recovery options that are much easier to setup than SQL Server.

When you use the Azure portal to create a SQL Database, the various plans under the pricing tier include three service tiers: Basic, Standard, and Premium.  Here are those three plans with their high-availability (HA) and disaster recovery (DR) options:

Basic: Automatic Backups, Point In Time Restore up to 7 days, Disaster recovery (DR): Geo-Restore, restore to any Azure region

Standard: Automatic Backups, Point In Time Restore up to 14 days, DR: Standard Geo-Replication, offline secondary

Premium: Automatic Backups, Point In Time Restore up to 35 days, DR: Active Geo-Replication, up to 4 online (readable) secondary backups

Here are more details on those options:

High Availability: Each database possesses one primary and two local replica databases stored on LRS Azure Blob Storage that reside in the same datacenter, providing high availability within that datacenter.  At least two of those databases are synchronous.  The hardware these databases reside on are on completely physically separate sub-systems.  So if the hardware fails, your database will automatically and seamlessly fail over to the synchronous copy.

Automatic Backups: All Basic, Standard, and Premium databases are protected by automatic backups.  Full backups are taken every week, differential backups every day, and log backups every 5 minutes.  The first full backup is scheduled immediately after a database is created.  Normally this completes within 30 minutes but it can take longer.  If a database is “born big”, for example if it is created as the result of database copy or restore from a large database, then the first full backup may take longer to complete.  After the first full backup all further backups are scheduled automatically and managed silently in the background.  Exact timing of full and differential backups is determined by the system to balance overall load.  Backup files are stored locally in blob storage in the same data center as your databases with local redundancy.  When you restore a database, the required backup files are retrieved and applied.  The full, differential, and log backups are also copied to the blob storage in the paired secondary region in the same geo-political area for disaster recovery purpose (RA-GRS).  These geo-redundant copies are used to enable geo-restore as explained shortly.

Point In Time Restore: Point In Time Restore is designed to return your database to an earlier point in time. It uses the database backups, incremental backups and transaction log backups that the service automatically maintains for every user database.  See Azure SQL Database Point in Time Restore.  To restore a database, see Recover an Azure SQL Database from a user error.  When you perform a restore, you’ll get a new database on the same server.

Geo-Restore: When you create a SQL Database server, you choose the region you want it in (i.e. East US), and this is your primary region.  If there is an incident in this region and a database is unavailable, you can restore it from the geo-redundant backup copy in the secondary region to any region, using the same technology as point in time restore, and therefore the databases are resilient to the storage outages in the primary region.  Note that with this option, your data could be up to one hour behind.  See Azure SQL Database Geo-Restore.

Standard Geo-Replication: This is where a copy of your data in the primary database is constantly being written asynchronously to a non-readable secondary database on a server in a different region (geo-redundancy).  In the event of a disaster you can fail over to the secondary.  Since the copy is asynchronous the data in the secondary database will be behind the primary, but not by more than five seconds (you can make the copy synchronous by using the system procedure sp_wait_for_database_copy_sync).  See Azure SQL Database Standard Geo-Replication.

Active Geo-Replication: Similar to Standard Geo-replication, your data is being asynchronously written except it’s on up to four secondary servers in different regions, and these secondaries are readable (each continuous copy is referred to as an called online secondary database).  You can also fail over to a secondary in the event of disaster in the same way as Standard Geo-Replication.  In addition, Active Geo-Replication can be used to support application upgrade or relocation scenarios without downtime, as well as load balancing for read-only workloads.  See Active Geo-Replication for Azure SQL Database.

A word about database failover:
If a region has an extended outage you will receive an alert in the Azure Portal and will see your SQL Database servers’ state set to Degraded.  At that point an application has a choice of initiating the failover or waiting for the datacenter to recover.  If your application needs to optimize for higher availability and can tolerate a data loss of 5 seconds then it should failover as soon as you receive an alert or detect database connectivity failures.  If your application is sensitive to data loss you may opt to wait for the SQL Database service to recover.  If this happens no data loss will occur.  In case you initiate the failover the database you must reconfigure your applications appropriately to connect to the new primary databases.  Once you have completed the failover you will want to ensure that the new primary is also protected as soon as possible.  Since primary region recovery may take time you will have to wait for your server to change from Degraded back to Online status. This will allow you to initiate geo-replication from the new primary to protect it.  Until seeding of the new secondary is completed your new primary will remain unprotected.

More info:

Creating a large data warehouse in Azure

Business Continuity Overview

Design for business continuity

SQL Database Enable Geo Replication in Azure Portal

Fault-tolerance in Windows Azure SQL Database

Distributed Storage: How SQL Azure Replicas Work

High Availability and Disaster Recovery for Azure SQL Databases

We’re proud to announce our lineup of speakers and sessions for SQL Saturday Chicago, taking place on Saturday, March 5, 2016!

Building a schedule for an event like this is never easy, especially given the number and variety of submissions. Thank you to all who submitted! We received a total of 148 abstracts from 66 different speakers representing 4 different countries and 23 U.S. states. While it would be amazing to include them all, there are neither enough hours in the day nor enough rooms at our venue to do so. The schedule contains 51 sessions from 51 different speakers representing 4 countries and 19 states.

Registration is open and filling quickly, but there’s still spots left as of when this post went live. Register now and get a day of free training, career development, and networking. (Onsite lunch is provided for $15.) Hope to see you there March 5th!

One more thing – we’ve also got pre-cons! We’re offering 4 amazing full-day pre-conference sessions on Friday, March 4. These in-depth workshops provide an amazing opportunity to study with top experts. Lunch, snacks, and coffee are provided with your pre-conference registration fee. See the event page for more details.

…and you should join me. 

On 2 February I’ll be speaking at TECHUnplugged Austin, Texas.  This event, which has free registration, focuses on how technology innovation is changing business and IT.

TECHunplugged is a full day conference focused on cloud computing and IT infrastructure.

Its innovative formula combines three essential parts of the industry for an exceptional exchange of information, insights and education:

– A group of independent, insightful and well-recognized influencers
– Leading disruptive technology vendors,
– End users who manage rich technology environments.

The ultimate goal of TECHUnplugged Conference is to bring quality information to IT decision makers by bringing them together with independent influencers and industry vendors, to engage, debate and be informed through open discussions on topics such as IT infrastructure, virtualization, cloud computing and storage.

I’m going to be talking about how data has changed over the years and how data quality issues can become obstacles to business innovation.

If you are in IT and would like to attend, use the registration form below.  If you use my special code, you’ll be entered to win a special prize of an Amazon Echo (I SO LOVE MINE!) at the event.

My promotional code is:

infoadvisors

Yes, all lowercase.

I hope to see you in Austin.  Maybe we can have tacos.

RapidMiner Overview

If you are searching for a data mining solution be sure to look into RapidMiner.  RapidMiner is an open source predictive analytic software that provides great out of the box support to get started with data mining in your organization.  They offer a free desktop software version to get you started.  The basic process is drag and drop and let’s you build out models and solutions very quickly.  You can integrate python and R if you have additional processing that you want to perform.

Sign up for a account and download the free version.  Once you have it downloaded.  Try out the 4 accelerators that are included: Direct Marketing, Predictive Maintenance, Churn, and Sentiment Analysis.  Each of these accelerators have sample datasets or allow you to use your own data to walk you through using the product.

RapidMiner – Homepage

Give it try and let me know how you like it.

The post What is RapidMiner appeared first on BI and Predictive Analytics.

Top 10 Tips for SQL Server Performance and Resiliency

This article is part 10 in a series on the top 10 most common mistakes that I have seen impact SQL Server Performance and Resiliency. This post is not all-inclusive.

Most common mistake #10: Storage

For the final post in the top 10 items that influence the performance and resiliency of the databases, we will talk about storage.  Storage is by far the number one cause of performance issues I have experienced, and over the last 20 years of my career, the changes with storage have been huge. When I first started as a professional, we were using internal spinning disks and then quickly changed up to internal RAID arrays. Five or so years later, we moved to external RAID cages.  Eventually the road took us to SAN and NAS storage and most recently SSD.  With all these changes, it is easy to see why we have administrators who focus on nothing but storage.  If you are fortunate enough to have a Storage Administrator, do you as a DBA still need to understand what the storage is doing?

How can you identify if you are having some sort of performance bottleneck?  There are a number of indicators that can provide you with the evidence your database is having a storage issue. A common indicator used for a number of years is the storage latency.  Storage latency information is collected by using the Performance Monitor in Windows.  Add the counters Average Disk/sec Read and Average Disk/sec Write.

The storage latency can be monitored in real time or the data can be recorded by starting a data collection.

According to Microsoft Best Practices, the latency on the disk the log file resides should be less than five milliseconds, and the data file latency should be less than 20 milliseconds.  In my experience, I have seen log file latency climb as high as 10 millisecond and sometimes a little higher during spikes without any end user impact.   In addition take location note of the TempDB database as we talked about in Tip 8. You will want to ensure you are keeping the TempDB on the fastest storage you can.

There are additional tools you may want to consider using to dig deeper into the performance of your storage systems such as SQLIO and SQLIOSIM both from Microsoft.  A couple popular third party tools include IOMeter and CrystalDiskMark.

Please do not miss my other blogs regarding this topic.

Top 10 Tips for SQL Server Performance and Resiliency

1. Improper Backups
2. Improper Security
3. Improper Maintenance
4. Not having a Baseline
5. SQL Server Max Memory
6. Change History
7. Disaster Recovery Plans
8. TempDB
9. AutoShrink

It’s a new year and I’ve given Thomas LaRock (@@sqlrockstar | blog ) a few months to recover and ramp up his training since our last Throwdown.  The trophies from all my wins are really cluttering my office and I feel back that Tom has not yet had a chance to claim victory.  So we will battling again in just a few days.

I’ll be dishing out the knowledge along with a handkerchief for Tom to wipe up his tears at SQL Saturday #461 Austin, TX on 30 January 2016.  This full day community-driven event features real database professionals giving free presentations on SQL Server and Data Platform topics.  All you need to do is register (again, it’s free) before all the tickets are gone.

Database Design Throwdown

Speaker(s):  Karen LopezThomas LaRock

Duration: 60 minutes

Track: Application & Database Development

Everyone agrees that great database performance starts with a great database design. Unfortunately, not everyone agrees which design options are best. Data architects and DBAs have debated database design best practices for decades. Systems built to handle current workloads are unable to maintain performance as workloads increase.Attend this new and improved session and join the debate about the pros and cons of database design decisions. This debate includes topics such as logical design, data types, primary keys, indexes, refactoring, code-first generators, and even the cloud. Learn about the contentious issues that most affect your end users and how to avoid them.

One of the other great benefits of attending these events is that you get to network with other data professionals who are working on project just like yours…or ones you will likely work on at some point. 

Join us an other data pros to talk about data, databases and projects. And make sure you give a #datahug to Tom after the Throwdown. He’s gonna need it.