<blockquote>
  <p>I want to be able to train a model that can classify text accurately. Surprisingly, I'm getting a high (>90%) for a relatively simple model like RandomForestClassifier just by using this summing up method. Any insights?</p>
</blockquote>

<p>If you look up papers on aggregating word embeddings you'll find out that this in fact occurs sometimes, especially if the texts are shorter.</p>

<blockquote>
  <p>What other way can I train my model? If I take every word and feed that into my model, how do I know how many words I should take? How do I input these words? In the form of a 2D array, where each word vector is a column?</p>
</blockquote>

<p>Have you tried keyword extraction? It can alleviate some of the problems with averaging</p>

<blockquote>
  <p>In doing so, have I destroyed any meaning the words might have
  possessed?</p>
</blockquote>

<p>As you remarked, you throw out information on word order. But that's not even the worst part: most of the times for longer documents if you embed everything the mean will get dominated by common words ("how", "like", "do" et c). BTW see my answer to <a href="https://stats.stackexchange.com/questions/312206/can-i-apply-word2vec-to-find-document-similarity/320657#320657">this question</a></p>

<p>Other than that, one trick I've seen is to average word vectors, but subtract first principal component of PCA on word embedding matrix. For details you can see for example <a href="https://github.com/PrincetonML/SIF" rel="nofollow noreferrer">this repo</a> which also links to the paper (BTW <a href="https://arxiv.org/pdf/1909.13494.pdf" rel="nofollow noreferrer">this paper</a> suggests you can ignore "Smooth Inverse Frequency" stuff since principal component reduction does the useful part).</p>
Does summing up word embedding vectors in ML destroy their meaning?
<p>For example, I have a paragraph which I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform quantity. </p>

<p>One thing I've done is taken every word in the paragraph, vectorized it using GloVe word2vec and then summed up all of the vectors to create a "paragraph" vector, which I've then fed in as an input for my model. In doing so, have I destroyed any meaning the words might have possessed? Considering these two sentences would have the same vector:
"My dog bit Dave" &amp; "Dave bit my dog", how do I get around this? Am I approaching this wrong?</p>

<p>What other way can I train my model? If I take every word and feed that into my model, how do I know how many words I should take? How do I input these words? In the form of a 2D array, where each word vector is a column?</p>

<p>I want to be able to train a model that can classify text accurately. 
Surprisingly, I'm getting a high (>90%) for a relatively simple model like RandomForestClassifier just by using this summing up method. Any insights?</p>

<p>Edit: One suggestion I have received is to instead featurize my data as a 2D array where each word is a column, on which a CNN could work. Another suggestion I received was to use transfer learning through the huggingface transformer to get a vector for the whole paragraph. Which one is more feasible?</p>