Sfrgttk

I am using the following command to write the output from a stored procedure to a file for further processing:

DECLARE @shellCommand VARCHAR(2000)

SET @shellCommand = 'SQLCMD -S ' + @@SERVERNAME + ' -d ' + @dbName + ' -U ' + @UserName + ' -P ' + @Password + ' -Q "' + @sqlCommand + '" -s "," -h -1 -k 1 -W -o "' + @outputfileName + '"'

@sqlCommand is set correctly, to "EXEC StoredProcedureName parameterValue" and when run independently apparently produces the correct data.

The problem I'm having is that one of the fields contains text greater than 256 characters long and is getting truncated when fed through the SQLCMD command.

On reading the documentation I see that the default column width is 256 characters. So I'm looking at using the -y or -Y option to allow more than 256 characters to be output. However, these options aren't compatible with the -W option which I was using to remove trailing spaces from the output. I started by using -y 0 (despite the performance warning - once I get the output working I can look at the performance). However, that produces some strange results.

The file consists of a header row followed by the data rows and should look something like this:

First,ABC,Number,String,ReallyLongString,...

A,0123,14.99,"Short string","Longish string",...

B,0456,23.99,"Normal string","Really, really long string that's causing problems",...

With the -W option the file format is correct but the really long string is truncated. The reason we found it was because the trailing " was missing and the file wasn't being read properly.

With -y 0 the really long string is getting output but large numbers of spaces are being added to apparently random columns. We're getting something like this:

First,ABC ,Number    [...]      ,String,ReallyLongString,...

A    ,0123,14.99     [...]      ,"Short string","Longish string",...

B    ,0456,23.99     [...]      ,"Normal string","Really, really long string that's causing problems",...

(There are many, many more spaces between "Number" and the next column as represented by the "[...]", I'm just showing a few).

There are more numeric values that have to be formatted in a similar way and it does appear that it's these that are causing the extra spaces after the value and before the next comma. We can live with a few extra spaces, but not as many as this as the resultant file is too large to be read by the target program.

The data is generated by a stored procedure that looks like this:

SELECT

    'First' AS First,

    'ABC' AS ABC,

    'Number' AS Number,

    'String' AS String,

    'ReallyLongString' AS ReallyLongString,

    ...



UNION ALL



SELECT

    'A' as First,

    Column1 as ABC,

    REPLACE(FORMAT(Column2, 'N2', 'en-GB'), ',', '') as Number, -- to get the format correct

    '"' + Column3 + '"' as String,

    '"' + Column4 + '"' as ReallyLongString,

    ....

FROM Table

WHERE <condition>

I'm thinking that the problem is the output from the stored procedure. I only added the -W option to the SQLCMD because of the extra trailing spaces being output, but looking at the stored procedure I can't work out where they are coming from. I changed the number formatting to include RTRIM:

RTRIM(REPLACE(FORMAT(Column2, 'N2', 'en-GB'), ',', ''))

but that appeared to make no difference.

Is there something I can do to the stored procedure or is there a combination of options to SQLCMD that will produce the desired output? Or am I going to have to find some other way of producing this file?

While trying out the various suggestions in the comments I have stumbled across a solution.

I replaced:

'"' + Column4 + '"' as ReallyLongString,

with:

'"' + REPLICATE('O', 4000) + '"' AS ReallyLongString,

and all of the text was output.

However, using a variable:

DECLARE @longText NVARCHAR(MAX)

SET @longText = REPLICATE('O', 4000);



'"' + @longText + '"' AS ReallyLongString,

didn't.

What did work was declaring the variable with a specific length:

DECLARE @longText NVARCHAR(4000)

So, as the text in this case is never going to be longer than 2000 characters (the device reading the file can't cope with text that long) I convert all the text to 2000 character long strings:

SELECT

    'A' as First,

    Column1 as ABC,

    REPLACE(FORMAT(Column2, 'N2', 'en-GB'), ',', '') as Number, -- to get the format correct

    '"' + CAST(Column3 as NVARCHAR(2000)) + '"' as String,

    '"' + CAST(Column4 as NVARCHAR(2000)) + '"' as ReallyLongString,

    ....

FROM Table

WHERE <condition>

This is the how. I still don't really know the why.

Strange... using -y 0 actually works perfectly for me. No white space padding or any other formatting issues. It actually works better than -W, even though the documentation doesn't explain why and still recommends using -W:

-W

This option removes trailing spaces from a column. Use this option together with the -s option when preparing data that is to be exported to another application. Cannot be used with the -y or -Y options.

It seems to me like -y 0 actually does what -W advertises, and more (unlimited string length). The only other option I had to use was -s ",".

I'm using version 13.1.0007.0 on Linux, so maybe the OP's issue is something that's been fixed in higher versions.

The only issue I encountered was a non-printable character (the box looking thing) being added at the end of very long strings (over 100K chars), but when the strings are quoted ('"' + field + '"'), the character goes away. ¯_(ツ)_/¯