The University of Edinburgh research team said its findings could help to shed light on the causes of depression, which affects about one in five people.
It could also pave the way for better diagnosis and treatments for people suffering from the condition, experts believe.
There is currently no test for diagnosing depression.
The condition has symptoms such as persistent low mood, losing interest in life and thoughts of self-harming.
Experts have suggested the condition may be a number of different illnesses with varying causes and effects.
Edinburgh researchers used samples provided by Generation Scotland, a bank of human tissue from volunteers, to look for genetic causes of depression.
They examined whether people known to have genes putting them at risk of schizophrenia shared a genetic make-up with those who had depression.
The research indicates there may be a distinct group of people with depression who have increased genetic risk for schizophrenia, the team concluded.
Schizophrenia is known to share some factors with depression, such as low mood and neuroticism.
The results suggest some people with depression have genetic factors associated with schizophrenia, researchers said.
Heather Whalley, senior research fellow in the University of Edinburgh's division of psychiatry, said: "Depression is a serious and disabling condition yet we know very little about what causes it.
"There appears to be a distinct form of depression based on the risk of schizophrenia.
"The more we know about the causes of different forms of depression, the more likely it is that we can develop better ways of diagnosing and treating what is a major cause of disability and trauma for so many people."
The research, part of an initiative called STRADL - stratifying resilience and depression longitudinally - is funded by the Wellcome Trust.
The findings are published in the journal Translational Psychiatry.
A type of depression may affect people who are genetically at risk of schizophrenia, a new study suggests.