def _process_links(self, links):
        links = unique_list(links, key=lambda link: link.url) if self.unique else links
        return links
Is called before the migration will be migrated
dataset/ETHPy150Open wcong/ants/ants/contrib/linkextractors/sgml.py/BaseSgmlLinkExtractor._process_links--crossover with--dataset/ETHPy150Open pboehm/django-data-migration/data_migration/migration.py/Migration.hook_before_all