Showing posts with label files. Show all posts
Showing posts with label files. Show all posts

Friday, March 30, 2012

Receiving errors when using foreach loop and excel connection manager...

Purpose: Need to import excel source data into SQL Server 2005 tables. Excel source data comes in nulitple excel files with the same structure but different data. I would appreciate someone taking a look at the following information and notifying me of what I am doing incorrectly.

I Inserted a foreach loop container, a data flow task located inside the foreach loop contaiiner, an excel and SQL Server 2005 connections.

After trying multiple times I went the following URL and followed step by step direction on how to connect excel workbooks dynamically: http://msdn2.microsoft.com/en-us/library/ms345182.aspx . I also used http://www.sqlstrings.com/ as a reference when creating the connection string.

Creating a Foreach Loop Container:

1. Opened foreach loop container 2.Set the Enumerator to 'Foreach File Enumerator" and configured the enumerator by setting the directory location and file base name to E:\Clients\Dep Comm\BEA\BEA_Test_Source and *PersonnelExpense*.xls respectively. 3. Clicked Variable Mapping; created two variables called, "ExcelFile", and "ExtProperties" and closed out of the foreach loop container.

I. Created Excel Connection:

  1. Created excel connection called, “Dynamic Excel Connection Manager,” that initially pointed to one of the excel workbooks.
  2. Went to the connection properties by right clicking the connection manager.
  3. Expanded Expressions and clicked the ellipsis button to bring up property expressions
  4. Chose Connection String in the Property.
  5. Clicked the Expression Ellipsis button.
  6. Put the following inside the Expression multi line text box:

A. "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +@.[User::ExcelFile] + ";Extended Properties=\"" + @.[User::ExtProperties] + "\""

  1. Clicked the Evaluate Expression button to get the following:
    1. Provider=Microsoft.Jet.OLEDB.4.0;Data Source=;Extended Properties=""
  2. Clicked Ok button
  3. Inserted a Data flow task inside the foreach loop container.

II. Configured Tasks that is associated with Dynamic Excel Connection Manager or Package:

  1. Set the Foreach loop container Delay Validation to true.
  2. Set the Data Flow Task Container Delay Validation to true.
  3. Set the Dynamic Excel Connection Manager Delay Validation to true.
  4. Set the SQL Server Connection Manager Delay Validation to true.
  5. Set the Package Delay Validation to true.
  6. Package Locale ID set to English

Ran the package after connecting the excel source data flow to the OLEDB destination and have inserted part of the error in this post. Please see below.

Error: 0xC0202009 at Package, Connection manager "Dynamic Excel Connection Manager": An OLE DB error has occurred. Error code: 0x80004005.

An OLE DB record is available.Source: "Microsoft JET Database Engine"Hresult: 0x80004005Description: "Could not find installable ISAM.".

I modified the connection string after receiving the error by removing the extended properties. The following is the modified connection string: "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +@.[User::ExcelFile]

I repeated step I.6 above and received the following expression: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=

I ran the package and received the following error in part: OLE DB record is available.Source: "Microsoft JET Database Engine"Hresult: 0x80004005Description: "Unrecognized database format 'E:\Clients\Dep Comm\BEA\BEA_Test_Source\PersonnelExpense_OCCs_051007.xls'."

I did not find anything helpful when I searched for the above errors and would very much appreciate anyone’s assistance on this issue as this issue needs to be taken care of ASAP.

Does anyone have any ideas as to why I received this error and what can I do to resolve this issue?

Your assistance in this matter is truly appreicated!

Thanks!!

Lee

Are there headers on the Excel spreadsheets?

The error is looking for a database format where the headers are matched as column headers.

If there are no headers, it doesnt' know how to match it.

Just my twist on it,

Adamus

|||

Hi Adamus,

I appreciate your feedback. To answer your question, the first row does contain column headers.The data flow task has an excel source file which uses the Dynamic Excel Connection manager that I mentioned in my original post. The excel source file connects to a derived column data task that creates derived columns and changes the data type to reflect that of the destination. The derived columns is then connected to the SQL Server destinationation table that matches the excel header columns to the destination table columns.

What should I do with the connection string that I mentioned in my first post? I have provided it here for you or anyone else to review.

"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +@.[User::ExcelFile] + ";Extended Properties=\"" + @.[User::ExtProperties] + "\""

Can you or anyone else tell me what I am doing wrong here?

Thanks!!

Lee

|||

Can you actually import at least one single file? forget about the foreach loop and the expression in the connection manager.

I just did a search on 'Could not find installable ISAM ' and found this:

http://support.microsoft.com/kb/209805

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=477902&SiteID=1

So, before going deeper, try to connect to a single execel file.

BTW, I just found a test package I created long time ago, and I see some differences with your approach:

I have only one variable in my for each loop container to hold the file name; and since I configured the loop container to retrive 'Fully Qualified" file name; the variable contains the path and the file name. In the excel connection manager I applied an expression to the ExcelFilePath property (not to the Connection String). The expression is just: @.{User::VarFileName] ; which is the variable I use in the loop container. I don't mess with the extended properties.sql

Wednesday, March 28, 2012

Receive multiple FTP files

Is there a way to get more than one file with a single ftp task in SQL 2005?

I need to get 5 files from one server. They are in two different directories is that makes any difference. Right now I have a separate task for each but would like to have one task if possible.

Thanks

You can use a wild card (*) in your path... If you can't use a wild card to get your 5 files, then you'll have to split up the tasks.|||

I figured something like that would be the answer...

Thats ok though... Thanks for the response.

|||

Bill, we actually use a table for our connection (directories, files, etc.) for FTP's and loop through the table replacing the FTP properties (in the FTP task) with the info from the table. If you did this, you could read the connection info into a recordset. Add a ForEach task and place the FTP task within the ForEach container.

The recordset info then drives the connection info for the FTP task... Since you mention two directories, you could use a row for each directory. Lots cleaner (and easier to maintain!) than having five separate FTP tasks firing...

sql

receive files from remote site only if file does not exist in local directory

I need to only receive files via ftp when the file does not exist on my local machine. FIles are being added to the remote location on a weekly basis and they are being downloaded locally. I do not want to download all the files each time. Instead, just download what was not already downloaded. Is there a way to do this? I want to do this using SSIS / ftp task.

Thanks in advance.

One way to accomplish this is to use the Execute Process task to run ftp.exe to get a list of the files on the server, then compare that list to your local directory. You could compare the file lists through a script task. Then FTP each file that you need to pull down.|||Thanks very much. I will try that.

Tuesday, March 20, 2012

Rebuild Index grew database .mdf files

Hello,

Overnight, I had a Rebuild Index job run and it grew the size of the database.mdf file approximately 4 times. 300MB to 1.2GB. It also changed the initial size of the database to 1.2GB. Is there any way to recover this space or to shrink the database?

I tried to shrink it with DBCC SHRINKDATABASE to no avail (only recovered 40MB).

Do I have any options to reclaim the space?

Any ideas?

Thanks.It turns out that the cause of the increase in size of database files was due to the fact that I had the Rebuild Index job set with a fill factor of 10%. I tested a bit and found that if I set it to reorganize pages with the default amount of free space that it didn't triple/quadruple/etc the size of the files.

Case Closed!

Rebooting SAN with SQL Data Files

Please help.
We are rebooting our SAN tonight and it has a lot of our Data and Log files
on it, including the master and msdb data files. Approximately 35 of our SQL
Servers are storing files on this SAN. Will SQL Server gracefully recover
from the reboot or will I need to stop the services prior to the reboot or is
there anything else I need to do? Thank you in advance for any help.
James
Are you asking if you can reboot the SAN while SQL Server is still running?
If so then I would not recommend that at all. You are likely to come up
with 35 db's in suspect mode. You should stop all the services to ensure no
one is writing to the data or log files when you reboot.
Andrew J. Kelly SQL MVP
"Hex Caliber" <hex@.caliber.com> wrote in message
news:C5EAC564-2231-46D2-BC59-B49627A757B6@.microsoft.com...
> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log
> files
> on it, including the master and msdb data files. Approximately 35 of our
> SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or
> is
> there anything else I need to do? Thank you in advance for any help.
> James
|||If at all possible I would recommend to shutdown any and all hosts
connected to your SAN before rebooting it. What kind of SAN is it? Do
you have multiple storage processors? Are you completely rebooting the
disk enclosures as well as the storage processors? You could consider
trespassing LUNs over and rebooting the storage processors in series
rather than all at once.
While I don't have any direct experience with yanking the drives out
from under SQL, I can't imagine it would be very good to do. You will
probably be facing data corruption lost transactions and serious data
inconsistency.
tM
Hex Caliber wrote:

> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log files
> on it, including the master and msdb data files. Approximately 35 of our SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or is
> there anything else I need to do? Thank you in advance for any help.
> James
|||Thank you, both of you. I'm not sure of the answers to TM's questions, but
it sounds like our plan will be to shut down the SQL Servers and then go
ahead with the SAN fix. Thanks again.
"Hex Caliber" wrote:

> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log files
> on it, including the master and msdb data files. Approximately 35 of our SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or is
> there anything else I need to do? Thank you in advance for any help.
> James
|||If I were you, shut the servers down, not just SQL Server.
The OS will not like it, and some NT Admins tend to put pagefiles on the
nice big SAN drives, resulting in your boxes blue screening when the drives
drop out from under the OS.
Rather safe than sorry....
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"Hex Caliber" <hex@.caliber.com> wrote in message
news:21B570FA-8773-4994-9A81-465243F59286@.microsoft.com...
> Thank you, both of you. I'm not sure of the answers to TM's questions,
but[vbcol=seagreen]
> it sounds like our plan will be to shut down the SQL Servers and then go
> ahead with the SAN fix. Thanks again.
> "Hex Caliber" wrote:
files[vbcol=seagreen]
our SQL[vbcol=seagreen]
recover[vbcol=seagreen]
or is[vbcol=seagreen]

Rebooting SAN with SQL Data Files

Please help.
We are rebooting our SAN tonight and it has a lot of our Data and Log files
on it, including the master and msdb data files. Approximately 35 of our SQ
L
Servers are storing files on this SAN. Will SQL Server gracefully recover
from the reboot or will I need to stop the services prior to the reboot or i
s
there anything else I need to do? Thank you in advance for any help.
JamesAre you asking if you can reboot the SAN while SQL Server is still running?
If so then I would not recommend that at all. You are likely to come up
with 35 db's in suspect mode. You should stop all the services to ensure no
one is writing to the data or log files when you reboot.
Andrew J. Kelly SQL MVP
"Hex Caliber" <hex@.caliber.com> wrote in message
news:C5EAC564-2231-46D2-BC59-B49627A757B6@.microsoft.com...
> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log
> files
> on it, including the master and msdb data files. Approximately 35 of our
> SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or
> is
> there anything else I need to do? Thank you in advance for any help.
> James|||If at all possible I would recommend to shutdown any and all hosts
connected to your SAN before rebooting it. What kind of SAN is it? Do
you have multiple storage processors? Are you completely rebooting the
disk enclosures as well as the storage processors? You could consider
trespassing LUNs over and rebooting the storage processors in series
rather than all at once.
While I don't have any direct experience with yanking the drives out
from under SQL, I can't imagine it would be very good to do. You will
probably be facing data corruption lost transactions and serious data
inconsistency.
tM
Hex Caliber wrote:

> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log file
s
> on it, including the master and msdb data files. Approximately 35 of our
SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or
is
> there anything else I need to do? Thank you in advance for any help.
> James|||Thank you, both of you. I'm not sure of the answers to TM's questions, but
it sounds like our plan will be to shut down the SQL Servers and then go
ahead with the SAN fix. Thanks again.
"Hex Caliber" wrote:

> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log file
s
> on it, including the master and msdb data files. Approximately 35 of our
SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or
is
> there anything else I need to do? Thank you in advance for any help.
> James|||If I were you, shut the servers down, not just SQL Server.
The OS will not like it, and some NT Admins tend to put pagefiles on the
nice big SAN drives, resulting in your boxes blue screening when the drives
drop out from under the OS.
Rather safe than sorry....
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"Hex Caliber" <hex@.caliber.com> wrote in message
news:21B570FA-8773-4994-9A81-465243F59286@.microsoft.com...
> Thank you, both of you. I'm not sure of the answers to TM's questions,
but[vbcol=seagreen]
> it sounds like our plan will be to shut down the SQL Servers and then go
> ahead with the SAN fix. Thanks again.
> "Hex Caliber" wrote:
>
files[vbcol=seagreen]
our SQL[vbcol=seagreen]
recover[vbcol=seagreen]
or is[vbcol=seagreen]

Rebooting SAN with SQL Data Files

Please help.
We are rebooting our SAN tonight and it has a lot of our Data and Log files
on it, including the master and msdb data files. Approximately 35 of our SQL
Servers are storing files on this SAN. Will SQL Server gracefully recover
from the reboot or will I need to stop the services prior to the reboot or is
there anything else I need to do? Thank you in advance for any help.
JamesAre you asking if you can reboot the SAN while SQL Server is still running?
If so then I would not recommend that at all. You are likely to come up
with 35 db's in suspect mode. You should stop all the services to ensure no
one is writing to the data or log files when you reboot.
--
Andrew J. Kelly SQL MVP
"Hex Caliber" <hex@.caliber.com> wrote in message
news:C5EAC564-2231-46D2-BC59-B49627A757B6@.microsoft.com...
> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log
> files
> on it, including the master and msdb data files. Approximately 35 of our
> SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or
> is
> there anything else I need to do? Thank you in advance for any help.
> James|||If at all possible I would recommend to shutdown any and all hosts
connected to your SAN before rebooting it. What kind of SAN is it? Do
you have multiple storage processors? Are you completely rebooting the
disk enclosures as well as the storage processors? You could consider
trespassing LUNs over and rebooting the storage processors in series
rather than all at once.
While I don't have any direct experience with yanking the drives out
from under SQL, I can't imagine it would be very good to do. You will
probably be facing data corruption lost transactions and serious data
inconsistency.
tM
Hex Caliber wrote:
> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log files
> on it, including the master and msdb data files. Approximately 35 of our SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or is
> there anything else I need to do? Thank you in advance for any help.
> James|||Thank you, both of you. I'm not sure of the answers to TM's questions, but
it sounds like our plan will be to shut down the SQL Servers and then go
ahead with the SAN fix. Thanks again.
"Hex Caliber" wrote:
> Please help.
> We are rebooting our SAN tonight and it has a lot of our Data and Log files
> on it, including the master and msdb data files. Approximately 35 of our SQL
> Servers are storing files on this SAN. Will SQL Server gracefully recover
> from the reboot or will I need to stop the services prior to the reboot or is
> there anything else I need to do? Thank you in advance for any help.
> James|||If I were you, shut the servers down, not just SQL Server.
The OS will not like it, and some NT Admins tend to put pagefiles on the
nice big SAN drives, resulting in your boxes blue screening when the drives
drop out from under the OS.
Rather safe than sorry....
--
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"Hex Caliber" <hex@.caliber.com> wrote in message
news:21B570FA-8773-4994-9A81-465243F59286@.microsoft.com...
> Thank you, both of you. I'm not sure of the answers to TM's questions,
but
> it sounds like our plan will be to shut down the SQL Servers and then go
> ahead with the SAN fix. Thanks again.
> "Hex Caliber" wrote:
> > Please help.
> >
> > We are rebooting our SAN tonight and it has a lot of our Data and Log
files
> > on it, including the master and msdb data files. Approximately 35 of
our SQL
> > Servers are storing files on this SAN. Will SQL Server gracefully
recover
> > from the reboot or will I need to stop the services prior to the reboot
or is
> > there anything else I need to do? Thank you in advance for any help.
> >
> > James

Friday, March 9, 2012

Reattaching old database

I have just installed SQLExpress version 9.0.1399.

I was using an earlier version of the server before and backed up a copy of my mdf files for safety's sake. However when I now try and attach the old databases I get an error that the database files from the old non-release version (590) is not compatible.

Probably my own fault - the information was probably clearly on display deep in a file somewhere in the documentation (cough). However I would kind of like to get at the data in these files.

Is there any way of getting hold of an earlier version of SQLExpress which will handle these files? Or can they be opened in any other way?

The full error message I am getting is:

Database 'myCMS' cannot be upgraded because its non-release version (590) is not supported by this version of SQL Server. You cannot open a database that is incompatible with this version of sqlservr.exe. You must re-create the database. (Microsoft SQL Server, Error: 950)

I have searched around for more information and the advice seems to be that I should open the database in an older copy of SQLExpress, dump the data with BCP and recreate the database.

So now my question is, where can I find installation files for earlier versions of SQLExpress as I don't seem to have a copy of the original executable?

|||I am afraid these are no longer available, you might be able to get them by calling Customer Support Services but this will cost money

Re-attaching databases after upgrading to SQL Server Enterprise Edition

Since Upgrading from SQL Standard Edition to SQL Server EE we have been unable to re-attach the database files that we detached before the upgrade, to SQL Server Management studio. We get an error message that reads:

TITLE: Microsoft SQL Server Management Studio

Failed to retrieve data for this request. (Microsoft.SqlServer.SmoEnum)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&LinkId=20476


ADDITIONAL INFORMATION:

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

CREATE FILE encountered operating system error 5(error not found) while attempting to open or create the physical file

Any help with this matter would be truly appreciated. Thank you.


You need to reset the NTFS permisions on the physical files. SQL 2005 locks out the physical files NTFS permissions when the databases are detached.

Reason for multiple filegroups/ files?

SQL2K
SP4
If my data is spread accross a Raid array, is there any real reason to use
multiple filegroups/ files? Will I see any performance benefit? Will I only
see it on VLDB's. If I do use them, is there a good way to group them
logically?
TIA, ChrisRChrisR
>Will I see any performance benefit?
Probably, if you put the filegroup on another PHYSICAL disk
>Will I only
> see it on VLDB's. If I do use them, is there a good way to group them
> logically?
Yes , it makes sence only for VLDB
Look, you will have to investigate it carefuly, I mean how you are going
to restore in case of failure or somethinmg like that , it is not so easy
to maintain and I have my doubt that nowadays we can really gain
perfiomance benefit from creating a new file group . Perhaps you might
think about adding a new datafile .NDF and seeing how it works
"ChrisR" <noFudgingWay@.NoEmail.com> wrote in message
news:%23hGlHLPKHHA.1912@.TK2MSFTNGP03.phx.gbl...
> SQL2K
> SP4
> If my data is spread accross a Raid array, is there any real reason to use
> multiple filegroups/ files? Will I see any performance benefit? Will I
> only
> see it on VLDB's. If I do use them, is there a good way to group them
> logically?
> TIA, ChrisR
>|||Add on ; please take a look into below URL's. But to make use of the full
advantage you may need multiple disk controller as Uri suggested.
http://www.databasejournal.com/features/mssql/article.php/1439801
http://www.informit.com/guides/content.asp?g=sqlserver&seqNum=42&rl=1
Thanks
Hari
"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:u4aAfSPKHHA.3936@.TK2MSFTNGP02.phx.gbl...
> ChrisR
>>Will I see any performance benefit?
> Probably, if you put the filegroup on another PHYSICAL disk
>>Will I only
>> see it on VLDB's. If I do use them, is there a good way to group them
>> logically?
> Yes , it makes sence only for VLDB
>
> Look, you will have to investigate it carefuly, I mean how you are going
> to restore in case of failure or somethinmg like that , it is not so easy
> to maintain and I have my doubt that nowadays we can really gain
> perfiomance benefit from creating a new file group . Perhaps you might
> think about adding a new datafile .NDF and seeing how it works
>
>
> "ChrisR" <noFudgingWay@.NoEmail.com> wrote in message
> news:%23hGlHLPKHHA.1912@.TK2MSFTNGP03.phx.gbl...
>> SQL2K
>> SP4
>> If my data is spread accross a Raid array, is there any real reason to
>> use
>> multiple filegroups/ files? Will I see any performance benefit? Will I
>> only
>> see it on VLDB's. If I do use them, is there a good way to group them
>> logically?
>> TIA, ChrisR
>>
>|||At the very least I always recommend creating one secondary filegroup to
place all the user objects and leave just the system objects in the primary
filegroup. This makes it cleaner and easier to do piecemeal restores if
needed. From there it depends on how you use your data and what your
hardware config is like. If you want to separate different groups of objects
such as Non-clustered indexes from the Clustered / tables onto different
drive arrays you need at least two filegroups. If you know this is something
you may do in the future but aren't now it still may make sense to separate
them into multiple FG's to make the move easier down the road. If the db is
relatively small, your hardware is decent you may never see any advantages
of multiple files or filegroups.
--
Andrew J. Kelly SQL MVP
"ChrisR" <noFudgingWay@.NoEmail.com> wrote in message
news:%23hGlHLPKHHA.1912@.TK2MSFTNGP03.phx.gbl...
> SQL2K
> SP4
> If my data is spread accross a Raid array, is there any real reason to use
> multiple filegroups/ files? Will I see any performance benefit? Will I
> only
> see it on VLDB's. If I do use them, is there a good way to group them
> logically?
> TIA, ChrisR
>|||Thanks everyone, this is all as I suspected.
"ChrisR" <noFudgingWay@.NoEmail.com> wrote in message
news:%23hGlHLPKHHA.1912@.TK2MSFTNGP03.phx.gbl...
> SQL2K
> SP4
> If my data is spread accross a Raid array, is there any real reason to use
> multiple filegroups/ files? Will I see any performance benefit? Will I
only
> see it on VLDB's. If I do use them, is there a good way to group them
> logically?
> TIA, ChrisR
>