#lang scribble/manual
@(require "defs.rkt")
@title[#:tag "creating-profiles" #:version apt-version]{Creating Profiles}
@apt-only{
Using @(tb), you can share your research artifacts with others by creating
your own profiles.
}
@clab-only{
In @(tb), a profile captures an entire cloud environment---the software
needed to run a particular cloud, plus a description of the hardware
(including network topology) that the cloud software stack should run on.
}
When you create a new profile, you are creating a new
@seclink["rspecs"]{RSpec}, and, usually, creating one or more
@seclink["disk-images"]{disk images} that are referenced by that RSpec. When
someone uses your profile, they will get their own
@seclink["experiments"]{experiment} that boots up the resources (virtual or
physical) described by the RSpec. It is common to create the resource specifification
for profiles using a @seclink["jacks"]{GUI} or by writing a
@seclink["geni-lib"]{python script} rather than dealing with RSpecs directly.
@section[#:tag "creating-from-existing"]{Creating a profile from an existing one}
The easiest way to create a new profile is by cloning or copying an existing one and
customizing it to your needs. The basic steps are:
When you @bold{clone} an experiment, you are taking an existing experiment, including a snapshot of the disk, and creating a new profile based on it. The new profile will be identical to the profile that experiment was based on in all other respects. Cloning only works on experiments with a single node.
If you @bold{copy} a profile, you are creating a new profile that is identical in every way to an existing profile. You may or may not have a running experiment using the source profile. And if you do have a running experiment, it does not impact the copy. After copying a profile, you can then modify it for your own use. And if you instantiate the copy, you can then take snapshots of disk images and use them in future version of your copy. Any profile that you have access to may be copied.
@subsection[#:tag "profile-creation-preparation"]{Preparation and precautions}
To create profiles, you need to be a @seclink["register"]{registered user}.
Cloning a profile can take a while, so we recommend that you
@seclink["experiments"]{extend your experiment} while creating it, and contact
us if you are worried your experiment might expire before you're done creating
your profile. We also strongly recommend testing your profile fully before
terminating the experiment you're creating it from.
When cloning, your home directory is @bold{not} included in the disk image snapshot! You will need to install your code and data elsewhere in the image. We recommend /local/. Keep in mind that others who use your profile are going to have their own accounts, so make sure that nothing in your image makes assumptions about the username, home directory, etc. of the user running it.
When cloning, be aware the only the contents of disk (not running process, etc.) are stored as part of the profile, and as part of the creation process, your node(s) will be rebooted in order to take consistent snapshots of the disk.
For the time being, cloning only works for single-node profiles; we will add support for multi-node profiles in the future.
When copying a profile, remember that the disk images of a currently running experiment are not saved. If you want to customize the disk images using copy, you must copy the profile first, then instantiate your copy, then take snapshots of the modified disk image in your experiment.
@apt-only{
If you want your profile to be usable by @seclink["guest-users"]{guest
users}, keep in mind that there are @seclink["guest-users"]{several
restrictions} placed on them; among these is the fact that they are not
allowed to make outgoing connections, meaning that the experiment must be
self-contained. For example, they will not be able to download software or
datasets from within the experiment---it should all be contained as part of
the profile.
}
@subsection[#:tag"cloning-a-profile"]{Cloning a Profile}
@itemlist[ #:style 'ordered
@instructionstep["Create an experiment"]{Create an experiment using the
profile that is most similar to the one you want to build. Usually, this
will be one of our facility-provided profiles with a generic installation
of Linux.}
@instructionstep["Set up the node the way you want it"]{Log into the node
and install your software, datasets, packages, etc. Note the
@seclink["profile-creation-preparation"]{caveat above} that it needs to be
installed somewhere outside of your home directory, and should not be tied
to your user account.}
@instructionstep["Clone the experiment to create a new profile"
#:screenshot "clone-button.png" ]{
While you are logged in, the experiment page for your active experiments
will have a ``clone'' button. Clicking this button will create a new
profile based on your running experiment.
Specifically, the button creates a copy of the @seclink["rspecs"]{RSpec}
used to create the experiment, passes it to the form used to create new
profiles, and helps you create a @seclink["disk-images"]{disk image} from
your running experiment.
}
@instructionstep["Create Profile"]{
You will be taken to a complete profile form and should fill it out as described below.
}
]
@subsection[#:tag"copying-a-profile"]{Copying a Profile}
@itemlist[ #:style 'ordered
@instructionstep["Choose a profile"
#:screenshot "begin-experiment.png" ]{
Find the profile you wish to clone using the “Start Experiment” selector. Then you can click “Show Profile” to clone the profile directly or you can instantiate the profile if you wish to create an experiment first. Both profiles themselves and active experiments can be copied.
}
@instructionstep["Copy the profile or experiment"
#:screenshot "copy-button.png" ]{
While logged in, both your experiment page and the show profile page will have a copy button. Clicking this button will create a profile based on that profile or experiment.
This button only copies the rspec or genilib script. No state in the active experiment is preserved.
}
@instructionstep["Create Profile"]{
You will be taken to a complete profile form and should fill it out as described below.
}
]
@subsection[#:tag "creating-the-profile"]{Creating the Profile}
After copying or cloning a profile (see above) or selecting the menu option to create a new profile from scratch, you will need to fill out the profile creation form in order to complete the creation process.
@itemlist[ #:style 'ordered
@instructionstep["Fill out information for the new profile"]{
After clicking on the ``clone'' button, you will see a form that
allows you to view and edit the basic information associated with your
profile.
@screenshot["create-profile-form.png"]
Each profile must be associated with a @seclink["projects"]{project}. If
you're a member of more than one project, you'll need to select which one
you want the profile to belong to.
Make sure to edit the profile's Description and Instructions.
The ``Description'' is the text that users will see when your profile is
listed in @(tb), when the user is selecting which profile to use. It is also
displayed when @seclink["sharing-profiles"]{following a direct link to your
profile}. It should give the reader a brief description of what they will
get if they create an experiment with this profile. If the profile is
associated with a paper or other publication, this is a good place to
mention that. @seclink["markdown"]{Markdown} markup, including hyperlinks,
are allowed in the profile description.
The ``Instructions'' text is displayed on the experiment page after the
user has created an experiment using the profile. This is a good place
to tell them where the code and data can be found, what scripts they
might want to run, etc. Again, @seclink["markdown"]{Markdown} is allowed.
The ``Steps'' section allows you to create a @seclink["tours"]{tour} of your
profile, which is displayed after a user creates an experiment with it.
This feature is mostly useful if your profile contains more than one
node, and you wish to explain to the user what the purpose of each node
is.
You have the option of making your profile usable to anyone, only
registered @(tb) users, or members of your project. Regardless of the
setting you chose here, @(tb) will also give you a
@seclink["sharing-profiles"]{direct link} that you can use to share your
profile with others of your choosing.
}
@instructionstep["Click ``Create''" #:screenshot "image-creating.png"]{
When you click the ``Create'' button, your node will be rebooted,
so that we can take a consistent snapshot of the disk contents. This
process can take several minutes or longer, depending on the size
of your disk image. You can watch the progress on this page. When
the progress bar reaches the ``Ready'' stage, your new profile is
ready! It will now show up in your ``My Profiles'' list.}
@instructionstep["Test your profile"]{Before terminating your experiment
(or letting it expire), we strongly recommend testing out the new profile. If
you elected to make it publicly visible, it will be listed in the profile
selection dialog on the front page of @url[(apturl)]. If not,
you can instantiate it from the listing in your ``My Profiles'' page. If
the profile will be used by guest users, we recommend testing it as one
yourself: log out, and instantiate it using a different username (you will
also have to use an alternate email address.}
@instructionstep["Share your profile"]{Now that your profile is working,
you can @seclink["sharing-profiles"]{share it} with others by sending
them direct links, putting links on your webpage or in papers, etc. See
``@secref["sharing-profiles"]'' for more details.}
]
@subsection[#:tag "updating-profiles"]{Updating a profile}
You can update the metadata associated with a profile at any time by going to
the ``My Profiles'' page and clicking on the name of the profile to go to the
profile page. On this page, you can edit any of the text fields (Description,
Instructions, etc.), change the permissions, etc.
@margin-note{As with cloning a profile, this snapshot feature currently only
works with single-node profiles.}
If you need to update the contents of the disk image in the profile, simply
create a new experiment from the profile. (You will only see this button on
experiments created from profiles that you own.) Once your experiment is ready,
you will see a ``Snapshot'' button on the experiment page. Log into your node,
get the disk changed the way you want, and click the button.
@screenshot["snapshot-button.png"]
This button kicks off the same image creation process that occurs during
@seclink["creating-from-existing"]{cloning a profile}. Once it's finished, any
new experiments created from the profile will use the new image.
As with creating a new profile, we recommend testing the profile before letting
your experiment expire. If something goes wrong, we do keep one previous image
file for each profile; currently, the only way to get access to this backup
is to @seclink["getting-help"]{contact us}.
@section[#:tag "jacks"]{Creating a profile with a GUI}
@(tb) embeds the Jacks GUI for simple creation of small profiles. Jacks can
be accessed by clicking the ``topology'' button on the profile creation or
editing page. Jacks is designed to be simple, and to ensure that the topologies
drawn can be instantiated on the @seclink["hardware"]{hardware available}.
Thus, after making certain choices (such as picking an operating system image)
you may find that other choices (such as the node type) become limited.
@screenshot["jacks-blank.png"]
Jacks has a ``palette'' on the left side, giving the set of node types
(such as physical or virtual machines) that are available. Dragging a node
from this palette onto the larger canvas area on the right adds it to
the topology. To create a link between nodes, move the mouse near the first
node, and a small black line will appear. Click and drag to the second node
to complete the link. To create a LAN (multi-endpoint link), create a link
between two nodes, then drag links from other nodes to the small grey box
that appears in the middle of the original link.
@screenshot["jacks-properties.png"]
To edit the properties of a node or link, select it by clicking on its icon
on the canvas. The panel on the left side will be replace by a property
editor that will allow you to to set the @seclink["disk-images"]{disk image}
for the node, set commands to be run when the node boots, etc. To unselect
the current node or link, and return to the palette on the left, simply
click a blank area of the canvas.
@section[#:tag "repo-based-profiles"]{Repository-Based Profiles}
You can turn any public @tt{git} repository (including those hosted on GitHub)
into a @(tb) profile. Simply place a @tt{geni-lib} script
named @tt{profile.py} into the top-level directory of your repository. When
you create a new profile, you can provide the URL for your repository. The URL
needs to be a @tt{http://} or @tt{https://} URL, and the CloudLab portal needs
to be able to clone the repository without authentication.
@margin-note{
Note that CloudLab is not a @tt{git} hosting service; while we do keep a
cache of your repository, we don't guarantee that the profile will continue
to work if the original repository becomes unavailable. We also have limits
on the size of the repositories that we will clone.
}
When you intantiate a repository-based profile, the repository will be cloned
into the directory @tt{/local/repository} on all nodes in the experiment. This
means that you can keep source code, startup scripts, etc. in your repository
and refrence them from @tt{profile.py}. @(tb) sets the @tt{pull} URL for all
of these clones to be your ``upstream'' repository, and attempts to set
a suitable @tt{push} URL for (it assumes that the hosting service uses @tt{ssh}
for pushes, and uses the @tt{git@"@":user/repo} convention). As a
result, @tt{git pull} and @tt{git push} should be connected to your repository.
There is an example repository on GitHub at
@hyperlink["https://github.com/emulab/my-profile"]{@tt{https://github.com/emulab/my-profile}}; if you don't already have a @tt{git} repository created, a good
way to get started is to for this one and crate a new profile pointing at your
fork.
@margin-note{
Pushing to your repository is still govered by the authentication and
permissions of your @tt{git} hosting service, so others using your profile
will not be able to push to your repository.
}
@subsection{Updating Repository-Based Profiles}
By default, the CloudLab profile does @bold{not} automatically update whenever
you push to your upstream repository; this means that people instantiating your
profile see the repository as it existed at the time @(tb) last pulled from it.
You can @bold{manually} cause @(tb) to pull from your repository using the
``Update'' button on the profile management page.
You can also set up @(tb) to @bold{automatically} pull from your repository
whever it is updating. To do so, you will need to set up a ``web hook'' on
the service that hosts your @tt{git} repository. @(tb) currently supports
webhooks for @tt{GitHub.com}, @tt{BitBucket.org}, and sites hosted using
GitLab (including both @tt{GitLab.com} and self-hosted GitLab installations.)
See the ``push URL'' in the Repository Info panel on the left side of the
profile page for the webhook
URL, and use the ``information'' icon next to it to get specific instructions
for setting webhook on each service. Once you have set the webhook up, every
time you push to your repository, your hosting service will let @(tb) know
that it should automatically initiate a pull. (This will not be instantaneous,
but should complete quickly in most cases.)
@subsection{Branches and Tags in Repository-Based Profiles}
By default, repository-based profiles will be instaniated from the @tt{master}
branch. At the bottom of the profile page, you will also find a list of all
branches and tags in the repository, and can instantiate the version contained
in any of them. Branches can be used for development work that is not yet ready
to be come the @tt{master} (default) version of the profile, and tags can be
used to mark specific versions of the profiles that were used for specific
papers or course assignments, for example.
@section[#:tag "creating-from-scratch"]{Creating a profile from scratch}
@(tb) profiles are described by
@hyperlink["http://groups.geni.net/geni/wiki/GENIExperimenter/RSpecs"]{GENI
RSpecs}. You can create a profile directly from an RSpec by using the
``Create Profile'' option from the ``Actions'' menu. Note that you cannot edit
the text fields until you upload an RSpec, as these fields edit (in-browser)
fields in the RSpec.
@section[#:tag "sharing-profiles"]{Sharing Profiles}
If you chose to make your profile publicly visible, it will show up in the main
``Select Profile'' list on @url[(apturl)]. @(tb) also gives you direct links to
your profiles so that you can share them with others, post them on your
website, publish them in papers, etc. The link can be found on the profile's
detail page, which is linked for your ``My Profiles'' page. If you chose to
make your profile accessible to anyone, the link will take the form
@code[(apturl "/p//")]. If you didn't make the profile
public, the URL will have the form @code[(apturl "/p/")], where
@code{UUID} is a 128-bit number so that the URL is not guessable. You can still
share this URLs with anyone you want to have access to the profile---for
example, to give it to a collaborator to try out your work before publishing.
@section[#:tag "versioned-profiles"]{Versioned Profiles}
Profiles are @italic{versioned} to capture the evolution
of a profile over time. When @seclink["updating-profiles"]{updating profiles},
the result is be a new version that does not (entirely) replace the profile
being updated.
When @seclink["sharing-profiles"]{sharing a profile}, you are given two links
to share. One link will take the user to the most recent version of the profile
that exists at the time they click the link. This is the most appropriate
option in most cases. There is also a link that takes one to a specific version
of the profile. This link is most useful for publication in papers or other
cases in which reproducability with the exact same environment is a concnern.