title: re.sub after matching. all instances of a repeated matching group, python
body: <p>I am matching digits in str using a regex in Python. My desire is to capture numbers that might have a thousands separator (for me, a comma or a space) or can just be a string of numbers. The following shows what my regex captures</p>
  
  <pre><code>&gt;&gt;&gt; import re
  &gt;&gt;&gt; test = '3,254,236,948,348.884423 cold things, ' + \
  '123,242 falling birds, .84973 of a French pen , ' + \
  '65 243 turtle gloves, 8 001 457.2328009 units, and ' + \
  '8d523c.'
  &gt;&gt;&gt; matches = re.finditer(ANY_NUMBER_SRCH, test, flags=re.MULTILINE)
  &gt;&gt;&gt; for match in matches:
  ...   print (str(match))
  ...
  &lt;_sre.SRE_Match object; span=(0, 24), match='3,254,236,948,348.884423'&gt;
  &lt;_sre.SRE_Match object; span=(27, 34), match='123,242'&gt;
  &lt;_sre.SRE_Match object; span=(37, 43), match='.84973'&gt;
  &lt;_sre.SRE_Match object; span=(46, 52), match='65 243'&gt;
  &lt;_sre.SRE_Match object; span=(55, 72), match='8 001 457.2328009'&gt;
  &lt;_sre.SRE_Match object; span=(73, 74), match='8'&gt;
  &lt;_sre.SRE_Match object; span=(75, 78), match='523'&gt;
  </code></pre>
  
  <p>This is the <em>matching</em> behavior I want. Now, I want to take each of the matched numbers and remove the thousands separators (<code>','</code> or <code>' '</code>) if they exist. This should leave me with</p>
  
  <pre><code>'3254236948348.884423 cold things, ' + \
  '123242 falling birds, .84973 of a French pen ,' + \
  '65243 turtle gloves, 8001457.2328009 units, ' + \
  'and 8d523c.'
  </code></pre>
  
  <p>Basically, I have one regex to capture the number. This regex is used in multiple places, e.g. to find dollar amounts, to get ordinal numbers, ... For this reason, I've named the regex, <code>ANY_NUMBER_SRCH</code>.</p>
  
  <p>What I want to do is something like the following:</p>
  
  <pre><code>matches = some_method_to_get_all_matches(ANY_NUMBER_SRCH)
  for match in matches:
    corrected_match = re.sub(r"[, ]", "", match)
    change_match_to_corrected_match_in_the_test_string
  </code></pre>
  
  <p>As things are, I can not use substitution groups. If you just want to see the regex, you can check out <a href="https://regex101.com/r/AzChEE/3" rel="nofollow noreferrer">https://regex101.com/r/AzChEE/3</a> Basically, part of my regex is as follows</p>
  
  <pre><code>r"(?P&lt;whole_number_w_thous_sep&gt;(?P&lt;first_group&gt;\d{1,3})(?P&lt;thousands_separator&gt;[ ,])(?P&lt;three_digits_w_sep&gt;(?P&lt;three_digits&gt;\d{3})(?P=thousands_separator))*(?P&lt;last_group_of_three&gt;\d{3})(?!\d)"
  </code></pre>
  
  <p>I'll represent that without the "scrolling line":</p>
  
  <pre><code>(r"(?P&lt;whole_number_w_thous_sep&gt;(?P&lt;first_group&gt;\d{1,3})"
    "(?P&lt;thousands_separator&gt;[ ,])"
    "(?P&lt;three_digits_w_sep&gt;(?P&lt;three_digits&gt;\d{3})"
    "(?P=thousands_separator))*"
    "(?P&lt;last_group_of_three&gt;\d{3})(?!\d)")
  </code></pre>
  
  <p>The regex engine doesn't keep the repeated <code>three_digits_with_separator</code> because of the <code>*</code> for repeated capturing groups.</p>
  
  <p>I'm sure there's a way to use the <code>span</code> parts of the <code>_sre.SRE_Match object</code>s. That would be quite involved, however, and I'm dealing with strings with thousands to hundreds-of-thousands of characters. <strong>Is there a simple way to do <code>re.sub</code> after the <code>re.match</code> or <code>re.iter</code> or whichever other method is used to find the number pattern?</strong></p>
  
  <p>@abarnert got me the right answer - using a lambda function. My comment under <a href="https://stackoverflow.com/a/51640754/6505499">@abarnert's answer</a>, beginning with 'Verified!' shows all the steps. Just in case that comment goes the way of the broken links, <a href="https://i.stack.imgur.com/wwFKZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wwFKZ.png" alt="here&#39;s an image of the comment, with all the solution steps."></a>.</p>
  
  <hr/>
  
  <h3>My Attempts</h3>
  
  <p>By the way, I have looked at these questions on SO (<a href="https://stackoverflow.com/questions/39213585/python-regular-expression-replacing-a-portion-of-match">replace portion of match</a>, <a href="https://stackoverflow.com/questions/1327369/extract-part-of-a-regex-match">extract part of a match</a>, <a href="https://stackoverflow.com/questions/30487380/replace-with-re-sub-after-matching-pattern">replace after matching pattern</a>, <a href="https://stackoverflow.com/questions/39155242/regex-not-matching-due-to-repeated-capturing-group-rather-than-capturing-a-repea">repeated capturing group stuff</a>), but they simply show how to use substitution groups. I've also tried to use <code>re.finditer</code> as shown below with the following result.</p>
  
  <pre><code>&gt;&gt;&gt; matches = re.finditer(lib_re.ANY_NUMBER_SRCH, test, flags=re.MULTILINE)     
  &gt;&gt;&gt; for match in matches:
  ...   print ("match: " + str(match))
  ...   corrected_match = re.sub(r"[, ]", "", match)
  ...   print ("corrected_match: " + str(corrected_match))
  ...
  match: &lt;_sre.SRE_Match object; span=(0, 24), match='3,254,236,948,348.884423'&gt;
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 3, in &lt;module&gt;
    File "/usr/lib/python3.6/re.py", line 191, in sub
      return _compile(pattern, flags).sub(repl, string, count)
  TypeError: expected string or bytes-like object
  &gt;&gt;&gt;   print ("corrected_match: " + str(corrected_match))
  </code></pre>
  
  <hr/>
  
  <h3>The Big Regex</h3>
  
  <p>In case something happens to the <a href="https://regex101.com/r/9vbHjn/1/" rel="nofollow noreferrer">regex101.com link</a>, here is the giant regex:</p>
  
  <pre><code>ANY_NUMBER_SRCH = r"(?P&lt;number_capture&gt;(?P&lt;pre1&gt;(?&lt;![^0-9,.+-])|)(?P&lt;number&gt;(?P&lt;sign_symbol_opt1&gt;(?&lt;![0-9])[+-])?(?P&lt;whole_number_w_thous_sep&gt;(?P&lt;first_group&gt;\d{1,3})(?P&lt;thousands_separator&gt;[ ,])(?P&lt;three_digits_w_sep&gt;(?P&lt;three_digits&gt;\d{3})(?P=thousands_separator))*(?P&lt;last_group_of_three&gt;\d{3})(?!\d)|(?P&lt;whole_number_w_o_thous_sep&gt;\d+))(?P&lt;decimal_separator_1&gt;[.])?(?P&lt;fractional_w_whole_before&gt;(?&lt;=[.])(?P&lt;digits_after_decimal_sep_1&gt;\d+))?(?P&lt;post1&gt;(?&lt;![^0-9,.+-])|)|(?P&lt;pre2&gt;(?&lt;![^0-9,.+-])|)(?P&lt;fractional_without_whole_before&gt;(?P&lt;sign_symbol_opt2&gt;(?&lt;![0-9])[+-])?(?P&lt;decimal_separator_2&gt;[.])(?P&lt;digits_after_decimal_sep_2&gt;\d+)))(?P&lt;post2&gt;(?&lt;![^0-9,.+-])|))"
  </code></pre>