Showing posts with label underlying. Show all posts
Showing posts with label underlying. Show all posts

Wednesday, March 7, 2012

performance of indexed views

I see their benefit -- trust me. one question I have is that it looks like my indexed view has to get updated each time the underlying base table changes. What happens if I do an insert or bcp into the underlying table -- will me table go offline while this data gets re-aggregated -- is there a way for me to schedule this? I see UDAs as well, and I think they are more flexible, but I'd be concerned that they are getting too far away from the optimizer and how are they refreshed -- recompile of the code?

Help..and I know it's marketing -- but what direction is the best for people that use the product, UDAs or Indexed Views.

Indexed views get updated in-line with table updates, just as if you basically had an additional index on the base table. So, if you perform an insert into the base table, it will not commit/finish until all indexes on the base table AND indexes from materialized views are updated. They basically are very similair in the way that the engine ensures additional indexes on the base table are kept updated. The same applies to how statistics are updated and managed, the same as the corresponding base table. An indexed view has an associated b-tree(s) structure backing it that must be kept up to date just as the base table is in real-time with data modifications.

As for Indexed Views vs. UDA's, they are 2 totally different technologies for different uses...I really don't see how the 2 would even compare. A UDA is calculated from an instruction and dataset you pass it at runtime, with no physical backing at all (unless you create for example an indexed computed column that is based on the UDA, but I won't go there). A UDA is basically the same as using any of the existing built-in aggregate functions in SQL Server (i.e. sum(), count(), min(), max(), etc.).

HTH

|||

The other thing to be aware of is that you always have to have a clustered index on an indexed view (this is what defines it) If you therefore have additional indexes on the indexed view, you will have the additional knock on impact when the underlying data is changed.

Indexed views are great in that they provide any easy way of providing the indexed view data quickly to the user with little effort, however you need to appreciate the impact on performance as well as the restrictions on how data is updated (SET statements etc).

|||I don't understand the 2nd part of the response you say they are different technologies, I agree, but they can be used for the same things no? If I wnatd to sum the sales of a bunch of regions and roll them up I could use an Indexed View for that -- aka summary table -- no? That's what I can do with a UDA? Help me here..|||Hi Chad -- I think there's something I just picked up on that I didn't relase -- that the Indexed View is a preset aggregate, while the UDA takes the data set and perform the aggregation dynamically, correct? If so, then is this just not a set of logic to run on the data set? Would you recommend a UDA over calling a SP to do the work? Do you see a lot of people using this, or would suggest it? What does it buy me -- just trying to understand how much CLR objects we want and should use based on the countless resources that say it's good for some stuff and not for others :)|||

Yes, that is correct. You could use either to achieve what you are attempting to get, however the biggest difference between the 2 technologies is that an indexed view is materialized on-disk, just like a table, and a UDA performs the aggregation on the data-set at time of request, like a query against a table.

A UDA is just like you mentioned, basically logic that is performed against the data set...the advantage of a UDA in 2005 is that you can create your own aggregates that don't already exist as pre-defined (i.e. sum(), min(), max())...for example, you could create a median() aggregate for example.

As for why you would use a UDA over a stored procedure would come down to a couple of things that would be different for many different scenarios, including performance, encapsulation, type of use, etc. For example, you could use a UDA just like you could use an existing pre-defined aggregate (i.e. within a select statement), like this:

select sum(column), myuda(column) from table

whereas you couldn't do the above with a stored procedure. Also, you may need to perform complex computational logic on the data, which the CLR would be better at. However, if you're just grabbing data, a stored procedure may be better...all would depend on the scenario.

HTH,

Performance of a billion-row table

Hello All!
I am building an application in VB.Net along with an underlying db in
SQL 2000. I am not too experienced with SQL performance issues and am
looking for some advice. I am about done designing the database (about
150 tables) and am looking at one table--the mother of all tables--that
will hold approximately 500 million rows more than any other table in my
db.
ISSUE:
I am wondering whether I need to break this table down into several
smaller tables or look into using a partitioned table. A lot of VB code
and SQL procedure code depends on how the data intended for this table
is ultimately stored.
BACKGROUND:
The table has four columns and is indexed (clustered) on all four
columns. (Can't reduce the # of indicies since this table is on the
"many" side of a one-to-many relationship with two other tables.) All
four columns hold numeric datatypes.
In the first year the related application is rolled out, this "main"
table will quickly grow to about 500 million rows. Over the next 9
years, the table will likely grow to about 1 billion rows.
Users will have on their laptops a "filtered" local/subscription copy of
the db which will replicate with the main publisher db (using an
anonymous pull subscription approach for replication). This "main"
table will hold about 200,000 to 500,000 rows of data in the "filtered"
subscription db.
Also, assume users' laptops have @.1.8ghz processors 512MB RAM. I don't
know about the server hardware specs other than this firm spends a lot
of cash on maintaining/building their computer infrastructure.
FOLLOW-UP QUESTIONS
Based on this structure, will the execution of SELECT, UPDATE, INSERT
and DELETE queries against the "main" table be noticebly slower (3
seconds or more) than the execution of *similar* queries against other
tables that hold only 20% to 30% as many rows?
When users are working directly off the publisher db on the server, will
queries against the "main" table perform just as fast as (if not faster
than) when they perform against the filtered "main" table in the
subscription db?
Thanks in advance for any advice!
Charles
*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!Charles,
That depends on how many rows you are going to return and what your WHERE
caluse will look like. By the way I don't get your comment:
> The table has four columns and is indexed (clustered) on all four
> columns. (Can't reduce the # of indicies since this table is on the
> "many" side of a one-to-many relationship with two other tables.) All
You can only have 1 clustered index. Are you saying you have the clustered
index as a compound index of all 4 columns and then you have an individual
non-clustered index on each column? What is the PK?
Can you post hte DDL for this table?
--
Andrew J. Kelly
SQL Server MVP
"Charles Wolfersberger" <cwolfersberger@.prodigy.net> wrote in message
news:u%23SMldlRDHA.2332@.TK2MSFTNGP10.phx.gbl...
> Hello All!
> I am building an application in VB.Net along with an underlying db in
> SQL 2000. I am not too experienced with SQL performance issues and am
> looking for some advice. I am about done designing the database (about
> 150 tables) and am looking at one table--the mother of all tables--that
> will hold approximately 500 million rows more than any other table in my
> db.
> ISSUE:
> I am wondering whether I need to break this table down into several
> smaller tables or look into using a partitioned table. A lot of VB code
> and SQL procedure code depends on how the data intended for this table
> is ultimately stored.
>
> BACKGROUND:
> The table has four columns and is indexed (clustered) on all four
> columns. (Can't reduce the # of indicies since this table is on the
> "many" side of a one-to-many relationship with two other tables.) All
> four columns hold numeric datatypes.
> In the first year the related application is rolled out, this "main"
> table will quickly grow to about 500 million rows. Over the next 9
> years, the table will likely grow to about 1 billion rows.
> Users will have on their laptops a "filtered" local/subscription copy of
> the db which will replicate with the main publisher db (using an
> anonymous pull subscription approach for replication). This "main"
> table will hold about 200,000 to 500,000 rows of data in the "filtered"
> subscription db.
> Also, assume users' laptops have @.1.8ghz processors 512MB RAM. I don't
> know about the server hardware specs other than this firm spends a lot
> of cash on maintaining/building their computer infrastructure.
> FOLLOW-UP QUESTIONS
> Based on this structure, will the execution of SELECT, UPDATE, INSERT
> and DELETE queries against the "main" table be noticebly slower (3
> seconds or more) than the execution of *similar* queries against other
> tables that hold only 20% to 30% as many rows?
> When users are working directly off the publisher db on the server, will
> queries against the "main" table perform just as fast as (if not faster
> than) when they perform against the filtered "main" table in the
> subscription db?
> Thanks in advance for any advice!
> Charles
> *** Sent via Developersdex http://www.developersdex.com ***
> Don't just participate in USENET...get rewarded for it!|||lets assume that your table has four numeric types of 9
bytes each for a total of 36 bytes per row
the overhead per row is 10-12 bytes, but the net row size
for 4 x 9 byte columns with a primary key clustered is ~45
bytes per row
this means 179 rows fit per page (99% fill)
then your index depth is:
depth rows
1 179
2 32,041
3 5,735,339
4 1,026,625,681
so your 500M-1B row table will have an index depth of 4,
you could increase the row capacity per index depth if you
did not make all 4 columns part of the primary key, but i
wouldn't worry about that right now
you are looking at ~22GB for the table, plus ~20GB per
index
I don't see the point of partitioning for the purpose of
reducing query cost. The size is not excessively large,
but you may consider using file groups
i would recommend creating the indexes with sort in
tempdb, so you don't burn space in the main db while
creating the indexes, meaning you will be more likely to
get contiguous space for each index
My inclination is to speculate that there is not a
substantial difference in the cost for a single row index
seek from a table with index depth 4 versus 2,
When selecting more than 1 row, cost of the subsequent
rows are the same regardless of the index depth
you should notice that insert/update/deletes will be take
~2X longer with 3-4 indexes compared with just the primary
key.
if know the number of rows queried, what types of joins
are involved, and platform, i can probably give you
estimates of query times
>--Original Message--
>Hello All!
>I am building an application in VB.Net along with an
underlying db in
>SQL 2000. I am not too experienced with SQL performance
issues and am
>looking for some advice. I am about done designing the
database (about
>150 tables) and am looking at one table--the mother of
all tables--that
>will hold approximately 500 million rows more than any
other table in my
>db.
>ISSUE:
>I am wondering whether I need to break this table down
into several
>smaller tables or look into using a partitioned table. A
lot of VB code
>and SQL procedure code depends on how the data intended
for this table
>is ultimately stored.
>
>BACKGROUND:
>The table has four columns and is indexed (clustered) on
all four
>columns. (Can't reduce the # of indicies since this
table is on the
>"many" side of a one-to-many relationship with two other
tables.) All
>four columns hold numeric datatypes.
>In the first year the related application is rolled out,
this "main"
>table will quickly grow to about 500 million rows. Over
the next 9
>years, the table will likely grow to about 1 billion rows.
>Users will have on their laptops a "filtered"
local/subscription copy of
>the db which will replicate with the main publisher db
(using an
>anonymous pull subscription approach for replication).
This "main"
>table will hold about 200,000 to 500,000 rows of data in
the "filtered"
>subscription db.
>Also, assume users' laptops have @.1.8ghz processors 512MB
RAM. I don't
>know about the server hardware specs other than this firm
spends a lot
>of cash on maintaining/building their computer
infrastructure.
>FOLLOW-UP QUESTIONS
>Based on this structure, will the execution of SELECT,
UPDATE, INSERT
>and DELETE queries against the "main" table be noticebly
slower (3
>seconds or more) than the execution of *similar* queries
against other
>tables that hold only 20% to 30% as many rows?
>When users are working directly off the publisher db on
the server, will
>queries against the "main" table perform just as fast as
(if not faster
>than) when they perform against the filtered "main" table
in the
>subscription db?
>Thanks in advance for any advice!
>Charles
>*** Sent via Developersdex http://www.developersdex.com
***
>Don't just participate in USENET...get rewarded for it!
>.
>