On his release Penry married a lady of Northampton, and lived there for some years. With the assistance of Sir Richard Knightley, he set up a printing press, which for nearly a year from Michaelmas 1588 was in active operation. It was successively located at East Moulsey (Surrey), Fawsley (Northamptonshire), Coventry and other places in Warwickshire, and finally at Manchester, where it was seized in August 1589. On it were printed Penry's "Exhortation to the governours and people of Wales, and View of... such publike wants and disorders as are in the service of God... in Wales"; as well as the celebrated Martin Marprelate tracts.