Sql 2016 Rebuild Master
Dec 27, 2018 - Rebuild Master Database in SQL Server Without Backup. At times, user delete some crucial information like linked servers, logins, SQL Server configuration, and other user objects. The master database is not coming online and the admin does not have a backup of the master database. Sqldba real time interview question on rebuild master database sql dba real time scenerio rebuild master database. SQL Server 2016 Part 3 - System Databases, Backing Up and Restoring.
Question Summary
A fragmented Clustered Index is not performing well, even after a REBUILD of the index. If the index is REORGANIZED then performance increases for the given table/index.
I am seeing this unusual behavior only on SQL Server 2016 and above, I have tested this scenario on different hardware and different versions (all personal machines and all have the conventional rotating hard disk). Let me know if need any more information.
Is this a bug in SQL Server 2016 and onwards?
I can provide the complete details and analysis with the script if anybody wants, but not providing right now because the script quite large and will take a lot of space in the question.
Please test the shorter version of sample script taken from the provided link below in your DEV environment if you have SQL Server 2016 and above.
SCRIPT
Results
Crystal Disk Mark Test Results
Detail
I am seeing an unusual behavior of storage engine (maybe) on SQL Server 2016 and above, I have created a highly fragmented table for (read problems with fragmentation) demo purpose then rebuilt it.
Even after rebuilding the index performance doesn't increase as expected.To make sure data access pattern should be in key order not in IAM-driven (Allocation Order Scan), I used the range predicate.
Initially, I thought maybe SQL Server 2016 and above is more aggressive for large scans. To check that, I adjusted the page count and row count but the performance pattern doesn't change. I tested all on a personal system so I can say there was no other user activity was going on.
I have tested this behavior on other hardware as well (all have traditional rotating hard disks). Performance patterns are almost same.
I have checked wait stats all seem normal only PAGELATCH_IO (using Paul Randal script) there. I checked data pages using DMV sys.dm_db_database_page_allocations it also seems ok.
If I reorganize the table or move all the data to a new table with same index definition disk IO performance increases. I have checked this with perfmon and it seems like reorganizing index /new table can leverage sequential IO and the rebuild index still using the random reads in spite of the fact that both have almost same data pages internal and external fragmentation.
I am attaching the complete query with the results on my system I captured.if you guys have SQL Server 2016 and above DEV box, please check this and share your results.
WARNING: This test consists some undocumented commands and DROPCLEANBUFFERS so not at all run at the production server.
If this is really a bug I think I should file it.
So the question is: Is it really a bug or I am missing something ;)
Links (pastebin)
Links:(Google Drive)
Paul White♦1 Answer
The queries in question exercise the SQL Server read-ahead feature. With read-ahead performance optimization, the SQL Server storage engine prefetches data during scans so that pages are already in buffer cache when needed by the query so less time is spent waiting for data during query execution.
The difference in execution times with read-alead reflects how well (or not) the storage system and Windows APIs handle large IO sizes along with differences in SQL Server read ahead behavior that vary by version. Older SQL Server versions (SQL Server 2008 R2 in the aforementioned article) limit prefeatch to 512K IO sizes whereas SQL Server 2016 and later issue read ahead IO in larger sizes to leverage capabilities of modern production grade commodity hardware (RAID and SSD). Keep in mind that SQL Server is generally optimized to run on current generation hardware at the time of release, exploiting larger processor cache, NUMA architecture, and storage system IOPS/bandwidth capability. Furthermore, Enterprise/Developer editions also perform prefetch more aggressively than lesser editions to maximize throughput even more.
To better understand the reason for the different performance of SQL 2008 R2 compared to later versions, I executed a modified versions of your scripts on a older physical machine with different versions of SQL Server Developer Edition. This test box has both 7200 RPM HDD and SATA SSD, allowing the same test to be run on the same machine against different storage systems and SQL versions. I captured file_read and file_read_completed events during each test with an Extended Event trace for more detailed analysis of IO and timings.
The results show roughly comparable performance with all SQL Server versions and storage system types except for SQL Server 2012 and later versions on a single HDD spindle following the clustered index rebuild. Interestingly, the XE trace showed 'Contiguous' mode during read-ahead scans in SQL Server 2008 R2 only; the trace showed 'Scatter/Gather' mode was used in all other versions. I can't say if this difference contributes to the faster performance.
Also, analysis of the trace data shows SQL 2016 issues much larger reads during read ahead scans and the average IO size varies by storage type. This doesn't necessarily mean SQL Server adjusts the read-ahead IO size based on physical hardware but instead that it might adjust the size based on unknown measurements. The heuristics used by the storage engine are not documented and may vary by version and patch level.
Below is a summary of test timings. I'll add more information gathered from the traces when I have some more time (unfortunately IO size is not available in SQL Server 2008 R2 XE). In summary, the IO profile differs by version and storage type. The average IO size for versions through SQL Server 2014 never exceeded 512K whereas SQL Server 2016 read over 4MB in a single IO in these tests. The number of outstanding reads was also much less in the SQL 2016 test because SQL Server fewer IO requests to accomplish the same work.
I also ran these tests on a VM with a HDD-backed SAN and observed similar performance as the SATA SSD. So the bottom line is this performance issue occurs only with a single-spindle HDD data file, something that is common only on PCs, not modern production systems. Whether or not this should be considered a performance regression bug is questionable but I'll reach out to see if I can get more information. Arma 3 respawn script multiplayer walkthrough.
EDIT
I reached out to my contacts and Erland Sommarskog pointed out the higher fragment count reported by sys.dm_db_index_physical_stats in later versions. Digging deeper, I noticed the REBUILD statement is a parallel query. The implication is that a parallel rebuild may actually increase the fragment count (and even introduce fragmentation on a table with no fragmentation) because space allocations are done in parallel. This will generally not affect performance except with read-ahead scans and is particularly a problem with singe-spindle spinning media as the tests show. This is a consideration in all SQL versions.
How To Rebuild Master Database In Sql Server 2017
Paul Randal pointed this out as by design and referenced this document for more information. A best practice for index rebuilds for a workload that will leverage read-ahead scans (e.g. data warehousing) is to rebuild WITH (MAXDOP = 1);
