The European Union Committee report has called on the government to ensure the Isle of Man, Jersey and Guernsey are "fully involved and engaged."
It also urges Parliament to remember its "unique constitutional responsibilities" to all three.
The Crown Dependencies are not part of the European Union or the UK so did not vote in the 2016 referendum.
However, the islands have a constitutional relationship both with the UK and the EU.
The report highlighted several potentially conflicting priorities which may arise for the islands during Brexit negotiations, including the maintenance of their centuries-old relationships with the UK and the retention of the benefits of their existing relationship with the EU.
It stated the Crown Dependencies' close constitutional, economic and cultural relationships with the UK "remain paramount", but added the implications of Brexit would be far-reaching and include an impact on the free trade, including fisheries and agriculture, both with the UK and EU.
It concluded that seeking to keep these priorities in balance would "not be easy" and the real test would come when negotiations began.
The Manx Chief Minister Howard Quayle said one of the Isle of Man's main concerns related to the freedom of movement of goods.
All three chief ministers were consulted by the committee and said they were "satisfied" with engagement so far.
A summit of EU member states to discuss Brexit is be held on 29 April, a month after the date given by Prime Minister Theresa May as when the UK will trigger Article 50.
The consequences of Brexit on the Crown Dependencies will be "significant", a House of Lords committee has concluded.