<p>It turns out this question was already asked on <a href="https://mathoverflow.net/questions/39159/isodiametric-hull">Math Overflow</a>, and people concluded it was likely to be a difficult problem. There are even some unanswered basic questions such as whether such a shape is unique.</p>

<p>So I don't have an exact solution, but hopefully this will get you closer or at least give you some ideas.</p>

<h2>Background</h2>

<p>For simplicity we can assume without loss of generality that the diameter of the initial polygon is 1.</p>

<p><a href="http://cdm.ucalgary.ca/cdm/index.php/cdm/article/download/190/126" rel="nofollow noreferrer">On a generalization of the Blaschke-Lebesgue theorem for disk-polygons</a> (M. Bezdek, 2009) describes a number of useful concepts. Relevant ones include:</p>

<ul>
<li>a disk-polygon is, informally, a convex set that forms a "fat" polygon where edges are replaced with arcs of curvature 1.</li>
<li>the set of points which we can add to a set of points <em>D</em> so that the resulting shape is of diameter at most 1 is called the dual disk-polygon <em>D*</em> of <em>D</em>.</li>
<li>the dual of the dual <em>D**</em> is called the spindle convex hull of <em>D</em>: it is the smallest disk-polygon containing <em>D</em>.</li>
</ul>

<p>Instead of working with polygons, it suffices to work with disk-polygons: we can always replace the original polygon with its spindle convex hull without changing the result.</p>

<p>We have that <em>D</em> ⊆ <em>D*</em> when <em>D</em> has diameter 1, and <em>D</em> = <em>D*</em> if and only if <em>D</em> has constant width 1. The solution <em>S</em> will have constant width 1 (although this is of course not sufficient). Therefore <em>D</em> ⊆ <em>S</em> if and only if <em>D</em> ⊆ <em>S</em> ⊆ <em>D*</em>: in particular, to approximate <em>S</em>, we only need to find a large enough disk-polygonal subset <em>D</em> of <em>S</em>. This is very powerful, because as we will see, saying that some point belongs or does not belong to <em>S</em> translates to both an upper bound <em>and</em> a lower bound on <em>S</em> (and therefore its area).</p>

<h2>Theoretical problems</h2>

<p>Ideally to find an efficient algorithm it would be useful to answer the following questions:</p>

<ul>
<li>is a globally optimal shape, i.e. a solution, necessarily unique?</li>
<li>is a locally optimal shape necessarily unique?</li>
<li>is the isodiametric hull of a polygon necessarily a circle of diameter 1 or a Reuleaux polygon of width 1?</li>
<li>if so, are the vertices of the Reuleaux polygon derived from finitely many unit-radius circle intersections, starting from the vertices of the original polygon?</li>
<li>is there a bound on the number of vertices of the Reuleaux polygon as a function of the number of vertices of the original polygon?</li>
</ul>

<p>Questions on the area of disk-polygons can be difficult: the problem solved in <a href="http://arxiv.org/pdf/math.MG/0108098" rel="nofollow noreferrer">Pushing disks apart - the Kneser-Poulsen conjecture in the plane</a> (K. Bezdek, R. Connelly, 2001) was a simple question regarding the area of intersections of disks in the plane which had remained unsolved for a long time.</p>

<h2>Practical(?) approaches</h2>

<p><strong>Global search</strong>:<br>
Start with the spindle convex hull of the polygon, and lazily construct an infinite search tree of increasing disk-polygons where each node partitions the set of all constant-width <em>X</em> satisfying <em>D</em> ⊆ <em>X</em> ⊆ <em>D*</em>, depending on whether some point <em>x</em> of <em>D*</em> \ <em>D</em> belongs or does not belong to <em>X</em>. The left branch is the spindle convex hull of <em>D</em> ∪ {<em>x</em>}. The right branch is the dual disk-polygon of <em>D*</em> ∩ {<em>y</em> : <em>x</em> ∉ [<em>y</em>, <em>z</em>] for all <em>z</em> in <em>D</em>}.</p>

<p>Unless you choose <em>x</em> very poorly (e.g. on the boundary of <em>D*</em> \ <em>D</em>), every infinite path of that tree should converge to a constant-width curve.</p>

<p>The idea is to explore the tree in a somewhat breadth-first way. Hopefully, if <em>x</em> is chosen in a sensible way, you will be able to discard all the branches where <em>D*</em> has a smaller area than the greatest area of a <em>D</em> found so far, as such branches cannot contain the solution. Then you will have a set of disk-polygons that converge to the set of solutions to the problem as you go deeper in the tree, hopefully while not growing too fast.</p>

<p>Some heuristics for <em>x</em> could be: take a point as close as possible to the inside of <em>D*</em> \ <em>D</em>, take a random point, and so on. It may also be interesting to incorporate some amount of depth-first search to have more precise lower bounds of the area of the solution which would allow to discard whole branches of the tree sooner.</p>

<p><strong>Local search</strong>:<br>
We could also work only with constant-width disk-polygons (Reuleaux polygons), and look at the effect of small deviations. But the search space is pretty large, so it's not clear how to do that.</p>
Smooth a convex polygonal shape so that it becomes as large as possible while retaining diameter
<p>Given a convex polygon, I am trying to grow its shape (as in "maximal area") while preserving its diameter. The diameter is defined as the length of the longest segment that can be placed within the polygon. Since the polygon is convex, I assume that this diameter can always be found by scanning all vertex pairs.</p>

<p>For example, given an equilateral triangle as an input polygon, the diameter of the triangle is the length of any edge; smoothing this would result in 3 circle segments as shown in the image<img src="https://i.stack.imgur.com/wVOvu.png" alt="before-and-after-smoothing"></p>

<p>For arbitrary convex polygons, a very inefficient algorithm is to compute the intersection of the maximum-diameter radius circles centered on each polygon vertex; this is what I am currently using (in Java). Is there anything better? Any pseudo-code or pointer to algorithm will be appreciated.</p>

<p>Another example: a squashed pentagon and its corresponding diameter-preserving maximal shape. The idea is that you cannot increase the area of this shape without increasing the diameter (that is, making it possible to draw a straight line within the bounds of the shape which is longer than the original diameter). In this particular case, it seems that a single circle with radius = polygon_diameter/2 (pink) is better than the intersection of multiple larger circles with radius = polygon_diameter (light-blue). The second image superimposes both areas to make comparison easier, but areas should completely enclose the polygon.</p>

<p><img src="https://i.stack.imgur.com/uFuGm.png" alt="enter image description here"></p>