title: import external data into hdfs: is edge node a bottle neck?
body: <p>I'm new to big data. I'm trying to ingest MongoDB collection into hdfs, and have choices of using mongoDB java API + HDFS java API, vs Spark MongoDB connector. </p>
  
  <ol>
  <li><p>How are both approaches different? I checked Spark MongoDB connector source code, it also uses mongoDB java API for write to mongoDB, but uses sparkSession for read from mongoDB.</p></li>
  <li><p>Since the MongoDB and HDFS cluster are different in our case, is the edge node acting as a staging area between MongoDB and HDFS cluster? If so, if the mongoDB collection too big, will edge node be a bottleneck? To generalize this question, when ingest external data sources (RDBMs, NoSQL etc) into HDFS, is the edge node always staging area?</p></li>
  <li><p>Follow-up to Q2. Is there a way to connect HDFS data node directly to mongoDB server, skipping the edge node? For example, from mongoDB sharding partition to HDFS partition.</p></li>
  </ol>
  
  <p>Appreciate your answer!</p>