xml


SSIS transform rows to columns from xml


I get an xml file from a web appliance stored on a shared file server. The xml is pooly structured. Here is an example:
<row>
<cell>2000-01-01T00:00:18.000000</cell>
<cell>2012-02-24T14:17:12.000000</cell>
<cell>0.18</cell>
<cell>3697.26464171841</cell>
<cell>3697.26464171841</cell>
<cell>3737.1</cell>
<cell>10.4318</cell>
<cell>0.229</cell>
</row>
Each cell element actually should be a different element. For example, if I were to contruct this xml sheet i would format it like this:
<run>
<startTime>2000-01-01T00:00:18.000000</startTime>
<EndTime>2012-02-24T14:17:12.000000</EndTime>
<effeicency>0.18</effeicency>
<good>3697.26464171841</good>
<bad>3697.26464171841</bad>
<average>3737.1</average>
<oee>10.4318</oee>
<up>0.229</up>
</run>
I am using SSIS XML source to pick this up and get the following:
Cell|row_id
2001-01-01|3
2001-01-01|3
0.18|3
3697.264|3
3697.264|3
3737.1|3
10.431|3
0.229|3
2012-01-01|4
2012-01-01|4
0.15|4
698.222|4
397.204|4
337.1|4
11.531|4
0.219|4
...
...
So my question is, can I use some transformation to pivot the data even thought I only have 2 columns in the source and I am not aggragating on anthing? My expected output would be
2001-01-01| 2001-01-01| 0.18| 3697.264| 3697.264| 3737.1| 10.431| 0.229
This solution may not be scalable for lots of records, but it work for me with these
kind of headaches. The key is to navigate the XML using XPath and convert the elements
to the appropriate datatype.
I don't know what the Primary Key would be I'm going to say ApplianceRowID for this example.
So the table structure would be something like this
CREATE TABLE [dbo].[WebApplianceStaging] (
ApplianceRowID [int] IDENTITY (1,1) NOT NULL,
RowData [nvarchar] (max) NULL
CONSTRAINT [PK_ApplianceRowID] PRIMARY KEY CLUSTERED
(
[ApplianceRowID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Write your SSIS import to bring in this data
then either in the SSIS or a stored proc convert the XML data to the appropriate
field types and insert into your final table.
Here I'm using a Common Table Expression and because of the number of decimal places I'm just using floats as the data types and initially making the dates varchar then converting to datetime with a left function and the 126 format specifier. Because it is XML there is a risk that the value is an empty string so you need to null those fields hence the nullif:
;WITH ApplianceRow(RowID, RowDataRaw) as
(
SELECT ApplianceRowID, RowData
FROM WebApplianceStaging
)
INSERT INTO [dbo].[FinalTable] --Your Final Table
([RowID],[StartTime],[EndTime],[Efficiency],[good],[bad],[average],[oee],[up])
SELECT
RowID
--,RowDataRaw
--,XML_Encoded
,CONVERT(
DATETIME
,LEFT(NULLIF(XML_Encoded.value('/row[1]/cell[1]','varchar(50)'),''),19)
,126
) AS StartTime
,CONVERT(
DATETIME
,LEFT(NULLIF(XML_Encoded.value('/row[1]/cell[2]','varchar(50)'),''),19)
,126
) AS EndTime
,NULLIF(XML_Encoded.value('/row[1]/cell[3]','float'),'') AS efficiency
,NULLIF(XML_Encoded.value('/row[1]/cell[4]','float'),'') AS good
,NULLIF(XML_Encoded.value('/row[1]/cell[5]','float'),'') AS bad
,NULLIF(XML_Encoded.value('/row[1]/cell[6]','float'),'') AS average
,NULLIF(XML_Encoded.value('/row[1]/cell[7]','float'),'') AS oee
,NULLIF(XML_Encoded.value('/row[1]/cell[8]','float'),'') AS up
FROM ApplianceRow
CROSS APPLY
(
SELECT XML_Encoded = CAST(RowDataRaw AS XML)
) as XML_Encoded

Related Links

Filtering XML nodes in memory without DOM?
How to evaluate an XML literal tag name in Scala?
xpath to get the entire xml tree structure
Grails Rest XML Rendering
XSD for repeat elements with same name but different attribute value
How to add tag text on one node as an attribute of another Tag..?
Get first child node in XSLT using local-name()
Using XML to obtain current date
Omit empty elements with groovy StreamingMarkupBuilder
Using properties file inside the xml
custom datatype - date and time
How to turn an XML file into SVG using XSL?
XML/WSDL Comparison tool
JQuery Mobile site works in Browser but not in Mobile Browser
Display Image in a Spark List in Flash Builder 4.5 from XML
How do you fetch a position based on a criteria match using XSLT?

Categories

HOME
notepad++
embedded
design-patterns
awk
apache-jena
numbers
serialization
fftw
appcelerator
access
newrelic
regular-language
modal-dialog
special-characters
rcloud
qpython
mql5
jsonpath
libc
favicon
scrolltop
bitnami
do-while
smartclient
poedit
pagespeed
hta
surveymonkey
progress-bar
linked-data
eviews
send
left-join
wacom
hreflang
joomla2.5
fossil
sim-toolkit
apic
lint
sql-server-ce
git-rewrite-history
metawidget
ogc
install.packages
openrasta
gitlist
spring-integration-sftp
reportservice2010
identification
filehandle
cmp
objectscript
jtree
structuremap4
slackware
node-request
mapxtreme
winusb
segment-io
watch-os-2
jodd
jboss-tools
nservicebus5
nupic
directdraw
avisynth
aspectj-maven-plugin
zend-currency
apache-spark-1.3
rapidsvn
memset
radiant
openargs
bin
unit-of-work
mongohq
beaker-testing
bjyauthorize
intersect
pstack
gobject-introspection
xcelsius
tmx
dojo-dnd
int64
movieplayer

Resources

Encrypt Message