title: (Python) XML to CSV conversion for DoD xccdf xml files
body: <p>I started writing a python script to take the XML output from the DISA SCAP Checking tool (SCC) and translate it to csv so that I can easily ingest it to Splunk for storage/analysis. Below is excerpt of the XML.</p>
  
  <pre><code>&lt;cdf:Benchmark id="xccdf_mil.disa.stig_benchmark_Mozilla_Firefox_Windows" style="SCAP_1.2" resolved="1" xsi:schemaLocation="http://checklists.nist.gov/xccdf/1.2 http://scap.nist.gov/schema/xccdf/1.2/xccdf_1.2.xsd http://cpe.mitre.org/dictionary/2.0 http://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.xsd http://cpe.mitre.org/language/2.0 http://scap.nist.gov/schema/cpe/2.3/cpe-language_2.3.xsd" xmlns:cdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dc="http://purl.org/dc/elements/1.1/"&gt;
        &lt;cdf:status date="2020-01-24"&gt;accepted&lt;/cdf:status&gt;
        &lt;cdf:title&gt;Mozilla Firefox Security Technical Implementation Guide&lt;/cdf:title&gt;
        &lt;cdf:description&gt;The Mozilla Firefox Security Technical Implementation Guide (STIG) is published as a tool to improve the security of Department of Defense (DoD) information systems. Comments or proposed revisions to this document should be sent via e-mail to the following address: disa.stig_spt@mail.mil&lt;/cdf:description&gt;
        &lt;cdf:notice id="terms-of-use"&gt;&lt;/cdf:notice&gt;
        &lt;cdf:reference href="https://cyber.mil"&gt;
              &lt;dc:publisher&gt;DISA&lt;/dc:publisher&gt;
              &lt;dc:source&gt;STIG.DOD.MIL&lt;/dc:source&gt;
        &lt;/cdf:reference&gt;
        &lt;cdf:plain-text id="release-info"&gt;Release: 1.5 Benchmark Date: 24 Jan 2020&lt;/cdf:plain-text&gt;
        &lt;cdf:platform idref="cpe:/a:mozilla:firefox:::~~~windows~~"&gt;&lt;/cdf:platform&gt;
        &lt;cdf:version update="http://iase.disa.mil/stigs"&gt;001.005&lt;/cdf:version&gt;
        &lt;cdf:metadata&gt;
              &lt;dc:creator&gt;DISA&lt;/dc:creator&gt;
              &lt;dc:publisher&gt;DISA&lt;/dc:publisher&gt;
              &lt;dc:contributor&gt;DISA&lt;/dc:contributor&gt;
              &lt;dc:source&gt;STIG.DOD.MIL&lt;/dc:source&gt;
        &lt;/cdf:metadata&gt;
        &lt;cdf:Profile id="xccdf_mil.disa.stig_profile_MAC-1_Classified"&gt;
              &lt;cdf:title&gt;I - Mission Critical Classified&lt;/cdf:title&gt;
              &lt;cdf:description&gt;&amp;lt;ProfileDescription&amp;gt;&amp;lt;/ProfileDescription&amp;gt;&lt;/cdf:description&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15768" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15771" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15772" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15774" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15775" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15776" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15778" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15779" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15983" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15985" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15986" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19742" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19743" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19744" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-64891" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-79053" selected="true"&gt;&lt;/cdf:select&gt;
        &lt;/cdf:Profile&gt;
        &lt;cdf:Profile id="xccdf_mil.disa.stig_profile_MAC-1_Public"&gt;
              &lt;cdf:title&gt;I - Mission Critical Public&lt;/cdf:title&gt;
              &lt;cdf:description&gt;&amp;lt;ProfileDescription&amp;gt;&amp;lt;/ProfileDescription&amp;gt;&lt;/cdf:description&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15768" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15771" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15772" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15774" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15775" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15776" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15778" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15779" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15983" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15985" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-15986" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19742" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19743" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-19744" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-64891" selected="true"&gt;&lt;/cdf:select&gt;
              &lt;cdf:select idref="xccdf_mil.disa.stig_group_V-79053" selected="true"&gt;&lt;/cdf:select&gt;
        &lt;cdf:Group id="xccdf_mil.disa.stig_group_V-15768" Id="xccdf_mil.disa.stig_group_V-15768"&gt;
              &lt;cdf:title&gt;FireFox Preferences – Verification&lt;/cdf:title&gt;
              &lt;cdf:description&gt;&amp;lt;GroupDescription&amp;gt;&amp;lt;/GroupDescription&amp;gt;&lt;/cdf:description&gt;
              &lt;cdf:Rule id="xccdf_mil.disa.stig_rule_SV-16707r1_rule" weight="10.0" Id="xccdf_mil.disa.stig_rule_SV-16707r1_rule" severity="medium"&gt;
                    &lt;cdf:version update="http://iase.disa.mil/stigs"&gt;DTBF050&lt;/cdf:version&gt;
                    &lt;cdf:title&gt;FireFox is configured to ask which certificate to present to a web site when a certificate is required.&lt;/cdf:title&gt;
                    &lt;cdf:description&gt;&amp;lt;VulnDiscussion&amp;gt;When a web site asks for a certificate for user authentication, Firefox must be configured to have the user choose which certificate to present. Websites within DOD require user authentication for access which increases security for DoD information. Access will be denied to the user if certificate management is not configured.&amp;lt;/VulnDiscussion&amp;gt;&amp;lt;FalsePositives&amp;gt;&amp;lt;/FalsePositives&amp;gt;&amp;lt;FalseNegatives&amp;gt;&amp;lt;/FalseNegatives&amp;gt;&amp;lt;Documentable&amp;gt;false&amp;lt;/Documentable&amp;gt;&amp;lt;Mitigations&amp;gt;&amp;lt;/Mitigations&amp;gt;&amp;lt;SeverityOverrideGuidance&amp;gt;&amp;lt;/SeverityOverrideGuidance&amp;gt;&amp;lt;PotentialImpacts&amp;gt;&amp;lt;/PotentialImpacts&amp;gt;&amp;lt;ThirdPartyTools&amp;gt;&amp;lt;/ThirdPartyTools&amp;gt;&amp;lt;MitigationControl&amp;gt;&amp;lt;/MitigationControl&amp;gt;&amp;lt;Responsibility&amp;gt;System Administrator&amp;lt;/Responsibility&amp;gt;&amp;lt;IAControls&amp;gt;&amp;lt;/IAControls&amp;gt;&lt;/cdf:description&gt;
                    &lt;cdf:reference&gt;
                          &lt;dc:publisher&gt;DISA&lt;/dc:publisher&gt;
                          &lt;dc:identifier&gt;205&lt;/dc:identifier&gt;
                          &lt;dc:type&gt;DPMS Target&lt;/dc:type&gt;
                    &lt;/cdf:reference&gt;
                    &lt;cdf:ident system="http://iase.disa.mil/cci"&gt;CCI-001274&lt;/cdf:ident&gt;
                    &lt;cdf:fixtext fixref="F-15985r1_fix"&gt;Set the value of "security.default_personal_cert" to "Ask Every Time".  Use the Mozilla.cfg file to lock the preference so users cannot change it.&lt;/cdf:fixtext&gt;
                    &lt;cdf:fix id="F-15985r1_fix"&gt;&lt;/cdf:fix&gt;
                    &lt;cdf:check system="http://oval.mitre.org/XMLSchema/oval-definitions-5"&gt;
                          &lt;cdf:check-content-ref name="oval:mil.disa.stig.mozilla.firefox.windows:def:20" href="U_Mozilla_Firefox_Windows_V1R5_STIG_SCAP_1-2_Benchmark-oval.xml"&gt;&lt;/cdf:check-content-ref&gt;
                    &lt;/cdf:check&gt;
              &lt;/cdf:Rule&gt;
        &lt;/cdf:Group&gt;
  &lt;cdf:TestResult id="xccdf_mil.disa.stig_testresult_scap_mil.disa.stig_comp_U_Mozilla_Firefox_Windows_V1R5_STIG_SCAP_1-2_Benchmark-xccdf.xml---xccdf_mil.disa.stig_profile_MAC-1_Classified-1" start-time="2020-04-28T16:34:31" version="001.005" end-time="2020-04-28T16:34:31" test-system="cpe:/a:spawar:scc:5.3"&gt;
              &lt;cdf:benchmark id="xccdf_mil.disa.stig_benchmark_Mozilla_Firefox_Windows" href="#scap_mil.disa.stig_comp_U_Mozilla_Firefox_Windows_V1R5_STIG_SCAP_1-2_Benchmark-xccdf.xml"&gt;&lt;/cdf:benchmark&gt;
              &lt;cdf:organization&gt;NIWC Atlantic&lt;/cdf:organization&gt;
              &lt;cdf:profile idref="xccdf_mil.disa.stig_profile_MAC-1_Classified"&gt;&lt;/cdf:profile&gt;
              &lt;cdf:target-id-ref name="SCC_DESKTOP-LAISPM9" href="" system="http://scap.nist.gov/schema/asset-identification/1.1"&gt;&lt;/cdf:target-id-ref&gt;
              &lt;cdf:platform idref="cpe:/a:mozilla:firefox:::~~~windows~~"&gt;&lt;/cdf:platform&gt;
              &lt;cdf:rule-result idref="xccdf_mil.disa.stig_rule_SV-16707r1_rule" weight="10.0" version="DTBF050" severity="medium" time="2020-04-28T16:34:31"&gt;
                    &lt;cdf:result&gt;fail&lt;/cdf:result&gt;
                    &lt;cdf:ident system="http://iase.disa.mil/cci"&gt;CCI-001274&lt;/cdf:ident&gt;
                    &lt;cdf:fix id="F-15985r1_fix"&gt;&lt;/cdf:fix&gt;
                    &lt;cdf:check system="http://oval.mitre.org/XMLSchema/oval-definitions-5"&gt;
                          &lt;cdf:check-content-ref name="oval:mil.disa.stig.mozilla.firefox.windows:def:20" href="#scap_mil.disa.stig_comp_U_Mozilla_Firefox_Windows_V1R5_STIG_SCAP_1-2_Benchmark-oval.xml"&gt;&lt;/cdf:check-content-ref&gt;
                    &lt;/cdf:check&gt;
              &lt;/cdf:rule-result&gt;
              &lt;cdf:rule-result idref="xccdf_mil.disa.stig_rule_SV-16710r3_rule" weight="10.0" version="DTBF105" severity="medium" time="2020-04-28T16:34:31"&gt;
                    &lt;cdf:result&gt;fail&lt;/cdf:result&gt;
                    &lt;cdf:ident system="http://iase.disa.mil/cci"&gt;CCI-000381&lt;/cdf:ident&gt;
                    &lt;cdf:fix id="F-15988r3_fix"&gt;&lt;/cdf:fix&gt;
                    &lt;cdf:check system="http://oval.mitre.org/XMLSchema/oval-definitions-5"&gt;
                          &lt;cdf:check-content-ref name="oval:mil.disa.stig.mozilla.firefox.windows:def:40" href="#scap_mil.disa.stig_comp_U_Mozilla_Firefox_Windows_V1R5_STIG_SCAP_1-2_Benchmark-oval.xml"&gt;&lt;/cdf:check-content-ref&gt;
                    &lt;/cdf:check&gt;
              &lt;/cdf:rule-result&gt;
  &lt;/cdf:TestResult&gt;
  &lt;/cdf:Benchmark&gt;
  </code></pre>
  
  <p>Here's what I've got thus far. It is only writing the values that I've specified in 'xccdf_head'. There are more values to extract, but I figure I'd like to get the format write before building.</p>
  
  <pre><code>import csv
  import xml.etree.ElementTree as ET
  
  # file to read
  tree = ET.parse("./test.xml")
  root = tree.getroot()
  
  # create file for writing
  xccdf_out = open('./test.csv', 'w')
  csvwriter = csv.writer(xccdf_out)
  xccdf_head = ['ID']
  csvwriter.writerow(xccdf_head)
  
  for group in root.findall('cdf:Benchmark'):
          row = []
          group_id = group.find('cdf:Group').find('cdf:title').text
          row.append(group_id)
          csvwriter.writerow(row)
  xccdf_out.close()
  
  </code></pre>
  
  <p>I am looking for an output that would capture the following from the two portions (cdf:Group and cdf:Result)</p>
  
  <p><code>'cdf:groupid', cdf:ruleid','cdf:ruleseverity','cdf:title','cdf:description','cdf:result'</code></p>
  
  <p>...so that ideally, it looks like this:</p>
  
  <p><code>xccdf_mil.disa.stig_group_V-79053, xccdf_mil.disa.stig_rule_SV-93759r3_rule, medium, Background submission of information to Mozilla must be disabled, VulnDiscussion&amp;gt;There should be no background submission(...),fail</code></p>