Showing posts with label event. Show all posts
Showing posts with label event. Show all posts

Friday, March 30, 2012

Putting reusable code into SSIS components....

If I have standard code for logging, event handling, error handling, etc...How to do you get the standard SSIS components to include this code...

I would hate to have to add this standard code, each time I add a component to my package...especialy if I have to build hundreds of them...

Thx in advance...

RC

You have a few options. The 2 worth considering are:

1) Write a custom log provider
2) Build your custom logging/error handling into eventhandlers.

2) is, I would say, easier. This demo may help: http://blogs.conchango.com/jamiethomson/archive/2005/06/11/1593.aspx
Oh, and don't forget about the ability to create template packages. Your template can contain all the logging/error handling functionality that you need which means you only have to define it once. See here for more on templates: http://blogs.conchango.com/jamiethomson/archive/2005/11/12/2380.aspx

Trust me, you don't have to define all this stuff every time you add a component/task.

-Jamie|||Note that BOL has a topic "Creating a Custom Log Provider in Integration Services" that has enough code to get you going. However at RTM there was not a separate standalone custom log provider code sample.

-Doug
|||What I've done is put source code for event handlers into a source control system, and then copy & paste them into every event handler box on every component. I admit, this is a TERRIBLE solution, but I've not had time to develop a XSLT solution -- the only decent solution I can think of is some XSLT to automate this terrible copy & paste. Plus I've feared automatically changing dtsx files, as they've changed through the beta & CTP cycle, and they seem to undergo many changes even with very small changes by a human.

Has anyone else solved this, or written XSLT they'd care to share, to automate the copy & paste into many copies of event handlers?

Granted, it would be nice if SSIS supported reusability, but, it seems to lack reusability at other levels as well (especially Derived Column expressions are a real problem lacking reusability), so I don't know how likely it is that they'll address this any time soon.
|||

Perry_Rapp wrote:

What I've done is put source code for event handlers into a source control system, and then copy & paste them into every event handler box on every component. I admit, this is a TERRIBLE solution, but I've not had time to develop a XSLT solution -- the only decent solution I can think of is some XSLT to automate this terrible copy & paste. Plus I've feared automatically changing dtsx files, as they've changed through the beta & CTP cycle, and they seem to undergo many changes even with very small changes by a human.

Has anyone else solved this, or written XSLT they'd care to share, to automate the copy & paste into many copies of event handlers?

Granted, it would be nice if SSIS supported reusability, but, it seems to lack reusability at other levels as well (especially Derived Column expressions are a real problem lacking reusability), so I don't know how likely it is that they'll address this any time soon.

Reusability will be addressed in a big way in the next version I am quite sure of that!

If you are logging to a database table then you could write a sproc to do the logging and call that from wherever you need to.

What I do is have a "master" package in which all my logging is configured. Then from that I call the packages that actually do the work. I pass in an ID and the "master" package looks up in a metadata DB which package(s) it needs to call for that ID.

Also be aware that you can build template packages that contain all your logging. http://blogs.conchango.com/jamiethomson/archive/2005/11/12/2380.aspx

-Jamie

Putting reusable code into SSIS components....

If I have standard code for logging, event handling, error handling, etc...How to do you get the standard SSIS components to include this code...

I would hate to have to add this standard code, each time I add a component to my package...especialy if I have to build hundreds of them...

Thx in advance...

RC

You have a few options. The 2 worth considering are:

1) Write a custom log provider
2) Build your custom logging/error handling into eventhandlers.

2) is, I would say, easier. This demo may help: http://blogs.conchango.com/jamiethomson/archive/2005/06/11/1593.aspx
Oh, and don't forget about the ability to create template packages. Your template can contain all the logging/error handling functionality that you need which means you only have to define it once. See here for more on templates: http://blogs.conchango.com/jamiethomson/archive/2005/11/12/2380.aspx

Trust me, you don't have to define all this stuff every time you add a component/task.

-Jamie|||Note that BOL has a topic "Creating a Custom Log Provider in Integration Services" that has enough code to get you going. However at RTM there was not a separate standalone custom log provider code sample.

-Doug
|||What I've done is put source code for event handlers into a source control system, and then copy & paste them into every event handler box on every component. I admit, this is a TERRIBLE solution, but I've not had time to develop a XSLT solution -- the only decent solution I can think of is some XSLT to automate this terrible copy & paste. Plus I've feared automatically changing dtsx files, as they've changed through the beta & CTP cycle, and they seem to undergo many changes even with very small changes by a human.

Has anyone else solved this, or written XSLT they'd care to share, to automate the copy & paste into many copies of event handlers?

Granted, it would be nice if SSIS supported reusability, but, it seems to lack reusability at other levels as well (especially Derived Column expressions are a real problem lacking reusability), so I don't know how likely it is that they'll address this any time soon.|||

Perry_Rapp wrote:

What I've done is put source code for event handlers into a source control system, and then copy & paste them into every event handler box on every component. I admit, this is a TERRIBLE solution, but I've not had time to develop a XSLT solution -- the only decent solution I can think of is some XSLT to automate this terrible copy & paste. Plus I've feared automatically changing dtsx files, as they've changed through the beta & CTP cycle, and they seem to undergo many changes even with very small changes by a human.

Has anyone else solved this, or written XSLT they'd care to share, to automate the copy & paste into many copies of event handlers?

Granted, it would be nice if SSIS supported reusability, but, it seems to lack reusability at other levels as well (especially Derived Column expressions are a real problem lacking reusability), so I don't know how likely it is that they'll address this any time soon.

Reusability will be addressed in a big way in the next version I am quite sure of that!

If you are logging to a database table then you could write a sproc to do the logging and call that from wherever you need to.

What I do is have a "master" package in which all my logging is configured. Then from that I call the packages that actually do the work. I pass in an ID and the "master" package looks up in a metadata DB which package(s) it needs to call for that ID.

Also be aware that you can build template packages that contain all your logging. http://blogs.conchango.com/jamiethomson/archive/2005/11/12/2380.aspx

-Jamie

Saturday, February 25, 2012

Publications with Pending Status

I am receiving the following errors in the event viewer:
Application log
SubSystem Message - Job 'BOS01-253-USICOAL-1'
(0x5E7E64490BAF924BB783E4C4F54C3E19), step 2 - SQL Server Agent could not
access the replication agent. Use the DCOMCNFG utility to confirm that the
SQL Server Agent Windows account has permissions to launch the replication
agent.
system log
Access denied attempting to launch a DCOM Server. The server is:
{08B0B2D9-3FB3-11D3-A4DE-00C04F610189}
The user is SYSTEM/NT AUTHORITY, SID=S-1-5-18.
I am running windows 2003 Server with XP Embedded clients, replication is
configured as push jobs from the distributer.
are you using remote agent activation?
If not, run DCOMCNFG to determine exactly where your merge agent is running.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Mark" <Mark@.discussions.microsoft.com> wrote in message
news:88C3A06B-CA0A-41D3-9100-4DD33BB79CA5@.microsoft.com...
>I am receiving the following errors in the event viewer:
> Application log
> SubSystem Message - Job 'BOS01-253-USICOAL-1'
> (0x5E7E64490BAF924BB783E4C4F54C3E19), step 2 - SQL Server Agent could not
> access the replication agent. Use the DCOMCNFG utility to confirm that the
> SQL Server Agent Windows account has permissions to launch the replication
> agent.
> system log
> Access denied attempting to launch a DCOM Server. The server is:
> {08B0B2D9-3FB3-11D3-A4DE-00C04F610189}
> The user is SYSTEM/NT AUTHORITY, SID=S-1-5-18.
> I am running windows 2003 Server with XP Embedded clients, replication is
> configured as push jobs from the distributer.
|||I don't thinks so, where do I check?
I have used DCOMCNFG and the merge agent is running locally.
Thanks
Mark
"Hilary Cotter" wrote:

> are you using remote agent activation?
> If not, run DCOMCNFG to determine exactly where your merge agent is running.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> "Mark" <Mark@.discussions.microsoft.com> wrote in message
> news:88C3A06B-CA0A-41D3-9100-4DD33BB79CA5@.microsoft.com...
>
>
|||Not sure if we are using remote agent activation, where do I check?
I have checked in DCOMCNFG and the merge agent is running locally.
thanks
Mark
"Hilary Cotter" wrote:

> are you using remote agent activation?
> If not, run DCOMCNFG to determine exactly where your merge agent is running.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> "Mark" <Mark@.discussions.microsoft.com> wrote in message
> news:88C3A06B-CA0A-41D3-9100-4DD33BB79CA5@.microsoft.com...
>
>
|||script out your publication and post it here.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Mark" <Mark@.discussions.microsoft.com> wrote in message
news:B3C9C4EA-A8EC-44F1-AFA5-FBF2CB721251@.microsoft.com...[vbcol=seagreen]
> I don't thinks so, where do I check?
> I have used DCOMCNFG and the merge agent is running locally.
> Thanks
> Mark
> "Hilary Cotter" wrote:
running.[vbcol=seagreen]
not[vbcol=seagreen]
the[vbcol=seagreen]
replication[vbcol=seagreen]
is[vbcol=seagreen]
|||Also check the Security tab for Microsoft SQL Server Replication Merge Agent
8.0 Properties, and verify that for the default Launch Permissions, the
everyone group has special access.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Mark" <Mark@.discussions.microsoft.com> wrote in message
news:7B60532F-1088-4BD9-9061-8BA6A7A044A7@.microsoft.com...[vbcol=seagreen]
> Not sure if we are using remote agent activation, where do I check?
> I have checked in DCOMCNFG and the merge agent is running locally.
> thanks
> Mark
> "Hilary Cotter" wrote:
running.[vbcol=seagreen]
not[vbcol=seagreen]
the[vbcol=seagreen]
replication[vbcol=seagreen]
is[vbcol=seagreen]
|||Here is one of the publications:
-- Enabling the replication database
use master
GO
exec sp_replicationdboption @.dbname = N'USICOAL', @.optname = N'publish',
@.value = N'true'
GO
use [USICOAL]
GO
-- Adding the transactional publication
exec sp_addpublication @.publication = N'PUB_ITEM', @.restricted = N'false',
@.sync_method = N'native', @.repl_freq = N'continuous', @.description =
N'Transactional publication of item data (PLU)', @.status = N'active',
@.allow_push = N'true', @.allow_pull = N'true', @.allow_anonymous = N'false',
@.enabled_for_internet = N'false', @.independent_agent = N'false',
@.immediate_sync = N'false', @.allow_sync_tran = N'false', @.autogen_sync_procs
= N'false', @.retention = 0, @.allow_queued_tran = N'false',
@.snapshot_in_defaultfolder = N'true', @.compress_snapshot = N'false',
@.ftp_port = 21, @.ftp_login = N'anonymous', @.allow_dts = N'false',
@.allow_subscription_copy = N'false', @.add_to_active_directory = N'false',
@.logreader_job_name = N'BOS01-253-USICOAL-1'
exec sp_addpublication_snapshot @.publication = N'PUB_ITEM',@.frequency_type =
4, @.frequency_interval = 1, @.frequency_relative_interval = 1,
@.frequency_recurrence_factor = 0, @.frequency_subday = 8,
@.frequency_subday_interval = 1, @.active_start_date = 0, @.active_end_date = 0,
@.active_start_time_of_day = 0, @.active_end_time_of_day = 235959,
@.snapshot_job_name = N'BOS01-253-USICOAL-PUB_ITEM-3'
GO
exec sp_grant_publication_access @.publication = N'PUB_ITEM', @.login = N'admin'
GO
exec sp_grant_publication_access @.publication = N'PUB_ITEM', @.login =
N'BUILTIN\Administrators'
GO
exec sp_grant_publication_access @.publication = N'PUB_ITEM', @.login =
N'distributor_admin'
GO
exec sp_grant_publication_access @.publication = N'PUB_ITEM', @.login = N'sa'
GO
-- Adding the transactional articles
exec sp_addarticle @.publication = N'PUB_ITEM', @.article =
N'ADDITIONAL_INFO', @.source_owner = N'dbo', @.source_object =
N'ADDITIONAL_INFO', @.destination_table = N'ADDITIONAL_INFO', @.type =
N'logbased', @.creation_script = null, @.description = null, @.pre_creation_cmd
= N'drop', @.schema_option = 0x00000000000000F3, @.status = 16,
@.vertical_partition = N'false', @.ins_cmd = N'CALL sp_MSins_ADDITIONAL_INFO',
@.del_cmd = N'CALL sp_MSdel_ADDITIONAL_INFO', @.upd_cmd = N'MCALL
sp_MSupd_ADDITIONAL_INFO', @.filter = null, @.sync_object = null,
@.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article = N'DEPT_SELL_RULE',
@.source_owner = N'dbo', @.source_object = N'DEPT_SELL_RULE',
@.destination_table = N'DEPT_SELL_RULE', @.type = N'logbased', @.creation_script
= null, @.description = null, @.pre_creation_cmd = N'drop', @.schema_option =
0x00000000000000F3, @.status = 16, @.vertical_partition = N'false', @.ins_cmd =
N'CALL sp_MSins_DEPT_SELL_RULE', @.del_cmd = N'CALL sp_MSdel_DEPT_SELL_RULE',
@.upd_cmd = N'MCALL sp_MSupd_DEPT_SELL_RULE', @.filter = null, @.sync_object =
null, @.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article = N'ITEM_GROUP',
@.source_owner = N'dbo', @.source_object = N'ITEM_GROUP', @.destination_table =
N'ITEM_GROUP', @.type = N'logbased', @.creation_script = null, @.description =
null, @.pre_creation_cmd = N'drop', @.schema_option = 0x00000000000000F3,
@.status = 16, @.vertical_partition = N'false', @.ins_cmd = N'CALL
sp_MSins_ITEM_GROUP', @.del_cmd = N'CALL sp_MSdel_ITEM_GROUP', @.upd_cmd =
N'MCALL sp_MSupd_ITEM_GROUP', @.filter = null, @.sync_object = null,
@.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article =
N'ITEM_GROUP_ITEM', @.source_owner = N'dbo', @.source_object =
N'ITEM_GROUP_ITEM', @.destination_table = N'ITEM_GROUP_ITEM', @.type =
N'logbased', @.creation_script = null, @.description = null, @.pre_creation_cmd
= N'drop', @.schema_option = 0x00000000000000F3, @.status = 16,
@.vertical_partition = N'false', @.ins_cmd = N'CALL sp_MSins_ITEM_GROUP_ITEM',
@.del_cmd = N'CALL sp_MSdel_ITEM_GROUP_ITEM', @.upd_cmd = N'MCALL
sp_MSupd_ITEM_GROUP_ITEM', @.filter = null, @.sync_object = null,
@.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article =
N'ITEM_GRP_ADDL_INFO', @.source_owner = N'dbo', @.source_object =
N'ITEM_GRP_ADDL_INFO', @.destination_table = N'ITEM_GRP_ADDL_INFO', @.type =
N'logbased', @.creation_script = null, @.description = null, @.pre_creation_cmd
= N'drop', @.schema_option = 0x00000000000000F3, @.status = 16,
@.vertical_partition = N'false', @.ins_cmd = N'CALL
sp_MSins_ITEM_GRP_ADDL_INFO', @.del_cmd = N'CALL sp_MSdel_ITEM_GRP_ADDL_INFO',
@.upd_cmd = N'MCALL sp_MSupd_ITEM_GRP_ADDL_INFO', @.filter = null, @.sync_object
= null, @.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article = N'PLU',
@.source_owner = N'dbo', @.source_object = N'PLU', @.destination_table = N'PLU',
@.type = N'logbased', @.creation_script = null, @.description = null,
@.pre_creation_cmd = N'drop', @.schema_option = 0x00000000000000F3, @.status =
16, @.vertical_partition = N'false', @.ins_cmd = N'CALL sp_MSins_PLU', @.del_cmd
= N'CALL sp_MSdel_PLU', @.upd_cmd = N'MCALL sp_MSupd_PLU', @.filter = null,
@.sync_object = null, @.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article = N'PLU_PRICE',
@.source_owner = N'dbo', @.source_object = N'PLU_PRICE', @.destination_table =
N'PLU_PRICE', @.type = N'logbased', @.creation_script = null, @.description =
null, @.pre_creation_cmd = N'drop', @.schema_option = 0x00000000000000F3,
@.status = 16, @.vertical_partition = N'false', @.ins_cmd = N'CALL
sp_MSins_PLU_PRICE', @.del_cmd = N'CALL sp_MSdel_PLU_PRICE', @.upd_cmd =
N'MCALL sp_MSupd_PLU_PRICE', @.filter = null, @.sync_object = null,
@.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article =
N'POS_ID_PLU_CODE', @.source_owner = N'dbo', @.source_object =
N'POS_ID_PLU_CODE', @.destination_table = N'POS_ID_PLU_CODE', @.type =
N'logbased', @.creation_script = null, @.description = null, @.pre_creation_cmd
= N'drop', @.schema_option = 0x00000000000000F3, @.status = 16,
@.vertical_partition = N'false', @.ins_cmd = N'CALL sp_MSins_POS_ID_PLU_CODE',
@.del_cmd = N'CALL sp_MSdel_POS_ID_PLU_CODE', @.upd_cmd = N'MCALL
sp_MSupd_POS_ID_PLU_CODE', @.filter = null, @.sync_object = null,
@.auto_identity_range = N'false'
GO
exec sp_addarticle @.publication = N'PUB_ITEM', @.article =
N'POS_ITEM_ADDL_INFO', @.source_owner = N'dbo', @.source_object =
N'POS_ITEM_ADDL_INFO', @.destination_table = N'POS_ITEM_ADDL_INFO', @.type =
N'logbased', @.creation_script = null, @.description = null, @.pre_creation_cmd
= N'drop', @.schema_option = 0x00000000000000F3, @.status = 16,
@.vertical_partition = N'false', @.ins_cmd = N'CALL
sp_MSins_POS_ITEM_ADDL_INFO', @.del_cmd = N'CALL sp_MSdel_POS_ITEM_ADDL_INFO',
@.upd_cmd = N'MCALL sp_MSupd_POS_ITEM_ADDL_INFO', @.filter = null, @.sync_object
= null, @.auto_identity_range = N'false'
GO
-- Adding the transactional subscription
exec sp_addsubscription @.publication = N'PUB_ITEM', @.article = N'all',
@.subscriber = N'TILL01-253', @.destination_db = N'USICOAL', @.sync_type =
N'automatic', @.update_mode = N'read only', @.offloadagent = 0,
@.dts_package_location = N'distributor'
GO
-- Adding the transactional subscription
exec sp_addsubscription @.publication = N'PUB_ITEM', @.article = N'all',
@.subscriber = N'TILL02-253', @.destination_db = N'USICOAL', @.sync_type =
N'automatic', @.update_mode = N'read only', @.offloadagent = 0,
@.dts_package_location = N'distributor'
GO
-- Adding the transactional subscription
exec sp_addsubscription @.publication = N'PUB_ITEM', @.article = N'all',
@.subscriber = N'TILL03-253', @.destination_db = N'USICOAL', @.sync_type =
N'automatic', @.update_mode = N'read only', @.offloadagent = 0, @.offloadserver
= N'<NULL>', @.dts_package_location = N'distributor'
GO
-- Adding the transactional subscription
exec sp_addsubscription @.publication = N'PUB_ITEM', @.article = N'all',
@.subscriber = N'TILL04-253', @.destination_db = N'USICOAL', @.sync_type =
N'automatic', @.update_mode = N'read only', @.offloadagent = 0, @.offloadserver
= N'TILL04-253', @.dts_package_location = N'distributor'
GO
"Hilary Cotter" wrote:

> script out your publication and post it here.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> "Mark" <Mark@.discussions.microsoft.com> wrote in message
> news:B3C9C4EA-A8EC-44F1-AFA5-FBF2CB721251@.microsoft.com...
> running.
> not
> the
> replication
> is
>
>