08/11/2026
by Marlon Ribunal
0 comments

When Do You Reach For A #temp Table?

Disclaimer: Rather than spending time writing a test scenario from scratch, I used Claude Code to generate the test scenarios for this article. I wanted a certain behavior to demonstrate a couple of differences between #temp and @table. You can download the test scripts from my public GitHub repo (see info at the bottom of this post).

#temp tables can be useful in so many ways. I hadn’t really given them much thought beyond using them when a situation called for it, until I saw the invitation for T-SQL Tuesday #201 Invitation: Temp Tables, Friend or Foe? This month’s T-SQL Tuesday is hosted by Jeff Taylor (b).

For this post, I needed to do some research because there is more to #temp tables than simply creating one and using it. As I started digging into the topic, I realized there are a lot of discussions around #temp tables and @table variables, especially around when one might be a better choice over the other.

One of the common arguments is that #temp tables go to disk while @table variables stay in memory. Another argument is that this is a myth. That got me curious about what the actual differences are and when they really matter. Discussions like this almost always starts with where the data is stored (disk vs memory).

Both #temp tables and @table variables use tempdb. Both allocate pages that are managed by the buffer pool. Depending on the size of the data, memory availability, and other factors, those pages may or may not ever be written to disk. If that’s the case, then maybe “memory versus disk” isn’t really the question we should be asking.

Let’s take a look at a couple of scenarios.

Test SQL: SQL Server 2022 CU25 (16.0.4255.1) container (Orbstack on Macbook pro)
CPU: 8 (schedulers)
MAXDOP: 0
Cost Threshold for Parallelism: Default 5 (for parallelism test scenario)

The difference is statistics

One of the biggest differences between #temp tables and @table variables is statistics. #temp tables can have statistics created and maintained by SQL Server, which gives the optimizer more information when generating an execution plan. @table variables do not have column statistics, although newer versions of SQL Server have improved their cardinality estimates through deferred compilation.

#temp tables can have statistics. Those statistics include histograms that help the optimizer understand how the data is distributed and make better estimates.

A table variable does not get column statistics. Even with newer versions of SQL Server and improvements like deferred compilation, the optimizer still does not have the same level of information that it has with a #temp table.

This is where many of the differences start to show up. Things like indexes, constraints, recompiles, and parallelism are often related to the fact that a #temp table is a temporary table with metadata that the optimizer can use, while a table variable has different behavior.

The rule I am starting to use from this research is simple:

Use a #temp table when the optimizer needs more information about the data. Use a table variable when the amount of data is small, the usage is simple, or you specifically need the behavior that a table variable provides.

Let’s take a look at a simple test.

I wanted to keep this as fair as possible. Both objects have the same structure, the same data, and the same nonclustered index. All the test scripts can be found in the Github repo (see link at the bottom of this post).

SQL
CREATE TABLE #temptable 
(
    id int NOT NULL, 
    grp int NOT NULL, 
    INDEX ix_grp NONCLUSTERED (grp)
);

DECLARE @tablevariable TABLE 
(
    id int NOT NULL, 
    grp int NOT NULL, 
    INDEX ix_grp NONCLUSTERED (grp)
);

The data is intentionally skewed. grp = 1 has 9,000 rows, while grp = 2 has only 10 rows.

When I checked the estimated row counts, the difference was obvious.

The #temp table had statistics, so the optimizer had information about the data distribution. The estimates were close to the actual row counts.

The table variable was different. Even though it had the same index, the optimizer did not have the same information available. The index gave it something to seek on, but it did not tell the optimizer how the data was distributed.

That distinction matters because estimates influence the rest of the execution plan. Row estimates affect join choices, memory grants, and other optimizer decisions. If SQL Server estimates 100 rows but the query actually returns 9,000 rows, the optimizer may choose a plan that isn’t optimal for the actual workload. That can also lead to memory spills to tempdb if the memory grant is too small.

One thing I noticed while testing this is that the estimate for a table variable is not always the same. For example, removing the index can change the estimate. The important part is that the optimizer still has limited information about the data inside a table variable.

So what about SQL Server 2019 and table variable deferred compilation?

Deferred compilation helps because SQL Server can see the table variable row count before compiling the statement. This improves cardinality estimates in many scenarios.

However, it does not create statistics. Knowing that a table variable has 10,000 rows is different from knowing how those 10,000 rows are distributed. A row count can help, but it does not replace a histogram.

sourcepredicateestimatedtrue
#tempgrp = 19,0009,000
#tempgrp = 21010
@tablevargrp = 11009,000
@tablevargrp = 210010

Table variables migh have a problem with parallelism

Note: It is quite complicated to test paralellism on my test environment. Some tests results returned “inconclusive”. Your mileage varies (depending on your config basically).

I also looked at how #temp tables and @table variables behave when it comes to parallelism.

A common statement is: “@table variables are always serial.” That may not be completely accurate. But again, I have not tested this extensively. FYI.

The restriction is related to modifying the table variable, not reading from it. For example, a regular SELECT or a query that reads from a table variable can still use parallelism. The limitation shows up when SQL Server is inserting into or modifying the table variable. I need to test this with bigger workload when I get the chance.

Again, this is not conclusive.

statementDOP
control: plain SELECT ... GROUP BY8
INSERT INTO #temp ... SELECT8
INSERT INTO @tablevar ... SELECT1
SELECT joining @tablevar8

Download the test scripts to see how this looks in your environment. Link to my Github repo is at the bottom of this post.

The impact really depends on how much data you are working with.

If you are inserting a small number of rows into a table variable and reading from it later, the lack of parallelism during the insert probably does not matter. The difference between a serial and parallel insert for a small dataset is not something you will likely notice.

Where this becomes important is when you start loading a large amount of data. A table variable that is used as a staging area for thousands or millions of rows can limit the insert operation to a serial plan when a #temp table could take advantage of parallelism.

For small amounts of data, use whichever option makes the code easier to understand. For larger data loads, a #temp table is usually worth considering.

The test scenario used in this post covers only a couple of the differences between #temp tables and @table variables. If you want to experiment with the scenarios yourself, you can grab the test scripts from my GitHub repository and run them in your own environment.

https://github.com/MarlonRibunal/sqlserver-demos/tree/main/tempdb

US Department of Defense STIG for SQL Server 2022

08/06/2026
by Marlon Ribunal
0 comments

SQL Server Security Hardening Guide Using the DoD STIG Checklist

Security on your SQL Server is important. That doesn’t need any explaining. But where do you start when evaluating the security of your SQL Server? If you are like me, and probably for many DBAs, that’s the hardest part. You know security matters, but without a structured baseline, it’s easy to overlook configuration issues that could expose your environment to unnecessary risk. Starting with a proven checklist gives you a clear way to identify gaps before they become problems.

And how do you even implement the principle of least privilege on the instance and database level?

Well, we’re lucky that the U.S. Department of Defense has provided a Security Technical Implementation Guide (STIG) for SQL Server, along with many other technologies. It gives you a well-established security baseline that you can use to evaluate your SQL Server environment, whether you’re working with an on-prem deployment, a virtual machine, or even a lab environment. Even if you’re not in a government-regulated organization, the STIG is still a practical reference for identifying security gaps and strengthening your SQL Server configuration.

You need two things: the SQL Server 2022 STIG and the STIG viewer.

Download the STIG file

Download the guide here. Search for SQL Server and select Microsoft SQL Server 2022 STIG. Click the Download button to, well, download the zipped STIG file.

You don’t have to extract the zip file. We’ll load the whole zip file on the viewer in the next step.

Install the STIG viewer

Download the STIG viewer here: https://www.cyber.mil/stigs/srg-stig-tools/. Download the msi installer called STIG Viewer 3.7.0-Win64 msi. Extract the installer package. Unfortunately, macOS is not supported. This is a Windows-only app.

Click Yes to allow the app to make changes to your device.

There is no wizard steps to follow. Upon clicking Yes, the STIG Viewer should be installed right away.

Load the STIG file

To load the STIG zip file (the first zip file we downloaded above) in the viewer, click the Open button in the STIG Viewer section (top panel). That will load both the STIG for SQL Server Instance and Database in the viewer.

You may want to add STIG to the Library so you don’t have to reload the STIG documentation each time. You can simply add the whole zip file.

If the STIG docs will not automatically appear after they are added in the library, close the app and open it again.

You can always go to the Dashboard by clicking the Home buttom at the upper right-hand navigation menu. Click the STIG Viewer button to open the STIG docs.

You can now then view the rules contained in the STIG docs. Select the Microsoft SQL Server 2022 Intance to view the rules for the instance (and Database for the database-level checks).

We will create our custom Checklist in our follow up post. We’ll also customize it so you can have a version of this STIG for your organization.

08/04/2026
by Marlon Ribunal
0 comments

How I Troubleshoot a Slow SQL Server Live

No two SQL Server environments are alike. A fix that resolves a performance issue in one environment may not work in another because every SQL Server instance has different workloads, hardware, data distribution, application behavior, and configuration. Even if you reproduce an issue in a test environment, there’s no guarantee the same solution will produce identical results in production.

So, probably instead of establishing a rigid step-by-step process, putting together a repeatable methodology can lead you to a resolution. This will eliminate blind guesses and at least bring you to an educated guess if the issue is too complicated.

A common workflow for troubleshooting performance issues on SQL Server may involve the following. But again, these are not necessarily phases or stages that you have to follow in a specific order. Real-world troubleshooting rarely happens in a perfect sequence. You may jump between these areas, revisit previous findings, or investigate multiple things simultaneously depending on the complexity of the issue.

  1. Define the scope
  2. Determine what changed
  3. Measure server health
  4. Identify bottlenecks
  5. Investigate the root cause
  6. Validate the fix / Monitor after implementation
  7. Document the incident

Scope it first

Every time I get a call about slow performance, the first thing I want to do is qualify the question itself. What does slow mean? Is it a widget that used to run for a few seconds and is now running for five minutes? By the way, this is one of the most common “slow” incidents I’ve seen in my last job at an ISV.

Scoping and discovery are probably the most critical parts of any troubleshooting task. This is the part where you can grasp the scope and nature of the issue at hand. Here, you will probably be able to establish your footing on how to approach the issue. Of course, a batch job running in SSIS or SQL Server Agent will require a different approach than, say, a process triggered by a user in the application’s frontend.

And, again, as I’ve said in my other post, I think it’s important to know when this issue started occurring. That alone might reveal a few important details about the very nature of the issue. Knowing whether you’re dealing with a single bad query or a server-wide issue will prevent you from investigating the wrong layer of the stack. For all you know, it’s not a performance issue at all, but a slowdown caused by the user’s ISP while working from home. I’ve seen this many times before.

My typical goal at the onset is to reproduce or replicate the issue in a test environment. That can give you the most bang for your buck because it allows you to investigate the issue and test potential solutions in a controlled environment.

The one thing we’re trying to prevent in the scoping is getting down the proverbial rabbit hole or chasing that red herring. Yes, I still love using those idioms in real life.

What changed?

Actually, this should be part of scoping, but I think it merits a whole section of its own. Knowing what changed could hold critical information as to the reason behind the issue. Maybe somebody just tweaked some lever on the application side, and that kind of caused a domino effect of some sort.

The main goal here is to find all the possible root causes and not point fingers at whoever might have been involved. There are a ton of reasons why a SQL Server might suddenly run slow. Configuration changes, new procedures, introduction of new indexes, unanticipated workload, network I/O, and even user workflows can cause slowdowns given the right circumstances where SQL Server can really slow down.

I’d start with the most recent change. And, of course, the typical answer to “What changed?” is almost always “nothing.” So, it’s up to you to determine how you surface what actually changed, if anything.

Know your baseline

If nothing has changed at all, then the next step is to determine any other activities that could be affecting the particular issue that you are investigating. It’s possible that the slowdown is caused by another task that is not necessarily related to the slowdown issue but is somehow affecting, for example, server resources.

I’ve seen cases where a third party started to send an amount of data that was not typical for an intra-day workload, which caused a domino effect on normal business operations. Not knowing what a “typical intra-day workload” looks like can complicate your investigation because you wouldn’t know if things are a factor in the issue or not.

For example, high CPU isn’t automatically a problem if the server normally runs at 90% CPU during month-end processing. While a query that normally consumes an infinitesimal amount of CPU and is suddenly taking 30% of the total CPU is almost certainly an issue. Always compare current metrics against a known baseline before concluding that something is abnormal.

Paul can tell you where it hurts

If you haven’t identified a possible root cause or figured out a narrative that may possibly connect the dots at this point, then it’s time to dig deeper. Paul Randal’s wait stats query can surface some clues about the ongoing issue. If you are troubleshooting a live issue, you probably want to combine that with Paul’s waiting task query. Glenn Berry’s Diagnostic queries are a must-have tool for any SQL Server DBA. If you are like me, when I encounter slow issues that are happening live, my instinct is to run Adam Machanic’s sp_whoisactive procedure. For many others running Brent O’s sp_BlitzFirst procedure is their go-to.

Wait stats and waiting tasks can be a superpower when it comes to troubleshooting a performance issue. Here is an example of how I used these two in the past to troubleshoot a live performance issue.

The wait type typically tells you where to go next. For example, blocking and lock waits send you to the head blocker (sp_WhoIsActive does a great job at identifying this). PAGEIOLATCH sends you toward storage I/O issues or queries performing large reads, possibly due to missing indexes. RESOURCE_SEMAPHORE points you toward memory grant pressure, while SOS_SCHEDULER_YIELD points toward CPU pressure. CXCONSUMER can be a benign wait type, but you should still investigate whether it is contributing to a bottleneck in a highly concurrent system.

Wait stats can help you identify the type of bottleneck SQL Server is experiencing, whether it’s related to I/O, CPU pressure, memory pressure, locking, or other resource contention.

Resource utilization

Ok, just like with wait stats, if you need to dig deeper, you should look into system resources as well. There is only so much you can achieve by optimizing indexes and queries. At some point, you need to ask whether the server has enough resources to handle the workload it is being asked to process. A perfectly tuned query can still struggle if the underlying infrastructure is already under pressure.

Here’s what I had to say about Disk IO on a LinkedIn post:

IOPS and Throughput (MB/s) in SQL Server

The key here is understanding whether you are dealing with a query problem, a workload problem, or a capacity problem. You can only go so far with index tuning and query optimization if the real issue is resource pressure underneath.

At some point, you have to understand what SQL Server is waiting on and whether the current infrastructure can support the workload. This is where troubleshooting becomes less about applying fixes and more about understanding the behavior of the system as a whole.

Bottleneck at the query level

We’re at the point where we are ruling out server-wide issues and closing in on specific queries that may be contributing to the problem. This is where tools like Query Store become extremely valuable because they allow you to look back at query behavior over time instead of relying only on what is happening at the exact moment of the incident.

One thing that I have been trying to learn lately is Query Store. At the time of writing, I am taking the SQLskills course on Query Store, IEQS. The more I learn about it, the more I realize how useful it can be when troubleshooting query performance issues, especially when dealing with plan regressions or queries that suddenly start behaving differently.

Of course, actual execution plans can give you a lot of information about where the bottleneck is occurring at the query level. Is it a cardinality estimation issue, memory grant pressure, spills, missing indexes, outdated statistics, or even parameter sniffing? The information you can glean from execution plans or Query Store can help you understand why a query is behaving the way it is and guide you toward the right tuning approach.

Is it blocking

If it’s a blocking chain, the first thing you want to do is find the head blocker and understand what it is doing before you start killing sessions. A blocked session is usually just a victim waiting for something else to finish. The real question is: what is holding the lock, and why is it taking so long?

This is where experience and judgment come into play. Just because you found a blocking session does not mean the immediate answer is to kill it. You need to understand what the session is doing, how long it has been running, and what impact terminating it will have. One interesting case I encountered in the past is a Service Broker queue that got stuck in limbo.

Communicate to stakeholders

During an incident, people understand that troubleshooting takes time. What they don’t want is uncertainty and silence while the issue is impacting the business. Keeping stakeholders informed shows that you are engaged, even when the root cause has not been identified yet.

In my experience, you don’t always have to provide a breakthrough every time you communicate. Sometimes a simple update like “we are seeing high I/O waits, we are checking storage latency, and we are looking at the queries involved” is enough to keep everyone aligned. The technical investigation may still be in progress, but at least everyone understands where things stand.

Validate the fix

Once the fix has been implemented, the work is not done yet. This is where you go back and validate if the change actually fixed the problem. Did the waits decrease? Did CPU return to its normal behavior? Did response times improve? Did the blocking clear? And probably the most important question: did the users notice the improvement?

One thing I always try to avoid is assuming that the issue is fixed just because the immediate symptom disappeared. A SQL Server environment is a complex system, and sometimes fixing one problem can expose another one. Keep monitoring after the change and make sure that things continue to behave as expected.

I’ve seen cases where a change looked successful during the initial troubleshooting session, only to find out later that the original problem came back or another issue surfaced. This is why validation and monitoring after the change are just as important as finding the fix in the first place.

Document the incident

Document what happened, what you checked, and what fixed the issue. Include the root cause, symptoms, timeline, diagnostics, fix, and any lessons learned. The next time a similar incident happens, this documentation can save you from starting the investigation from scratch.

Effective SQL Server troubleshooting is less about having the perfect fix ready and more about knowing how to narrow down the possibilities. You start with what you know, gather evidence, validate your assumptions, and slowly eliminate what doesn’t fit.

In my experience, the DBAs who troubleshoot well are not the ones who have memorized every possible solution. They are the ones who know how to ask the right questions, understand what the system is telling them, and make changes based on evidence. Having a repeatable methodology will take you much further than a collection of random tuning techniques.

Well, take it with a grain of salt. It depends. It always is.

Verified by MonsterInsights