A Fused Large Language Model for Predicting Startup Success
Abdurahman Maarouf Stefan Feuerriegel Nicolas Pröllochs




















Maarouf, Feuerriegel, and Pröllochs









A Fused Large Language Model for Predicting Startup Success






A Fused Large Language Model for Predicting Startup Success









Abdurahman Maarouf
LMU Munich & Munich Center for Machine Learning, a.maarouf@lmu.de, 
Stefan Feuerriegel
LMU Munich & Munich Center for Machine Learning, feuerriegel@lmu.de, 
Nicolas Pröllochs
Justus Liebig University Giessen, nicolas.proellochs@wi.jlug.de 


 


 
Investors are continuously seeking profitable investment opportunities in startups and, hence, for effective decision-making, need to predict a startup's probability of success. Nowadays, investors can use not only various fundamental information about a startup (e.g., the age of the startup, the number of founders, and the business sector) but also textual description of a startup's innovation and business model, which is widely available through online venture capital (VC) platforms such as Crunchbase. To support the decision-making of investors, we develop a machine learning approach with the aim of locating successful startups on VC platforms. Specifically, we develop, train, and evaluate a tailored, fused large language model to predict startup success. Thereby, we assess to what extent self-descriptions on VC platforms are predictive of startup success. Using 20,172 online profiles from Crunchbase, we find that our fused large language model can predict startup success, with textual self-descriptions being responsible for a significant part of the predictive power. Our work provides a decision support tool for investors to find profitable investment opportunities. 




Machine Learning; Text Mining; Large Language Models; Deep Learning; Venture Capital


Wind turbine condition monitoring based on intra- and inter-farm federated learning
    [
    Accepted XXX. Received YYY; in original form ZZZ
===================================================================================

=10000






§ INTRODUCTION





Startups are ventures undertaken by entrepreneurs to seek, develop, and validate a business model <cit.>. For investors, startups represent investment opportunities with substantial financial risks yet often also with the prospect of large returns. Return on investment can easily exceed the initial investment by several orders of magnitude. As an example, the early-stage investment of Peter Thiel of 0.5 million USD into Facebook increased in value by more than 1 billion USD <cit.>. However, successful investments into startups are rare. Many startups cannot establish an economic business model and eventually fail <cit.>. Given the high failure rate among startups, investors are confronted with the non-trivial decision-making task of identifying startups that will eventually be successful <cit.>.




In order to find successful startups, investors can nowadays access information about startups through online platforms for venture capital (VC). A prominent example is Crunchbase, where startups can present their venture to investors through a detailed online profile. The online profiles can include both (i) fundamental variables, which provide structured information on founders, funding, and the business sector, and (ii) textual self-description. The latter is a free text that can be used to describe the startup in verbal form. Startups can use such online profiles to inform about the venture's prospects and attract the interest of venture capitalists and other potential investors <cit.>.


Prior literature has explored the potential of leveraging VC platform data (e.g., from Crunchbase) to predict startup success due to their comprehensive coverage <cit.>. However, prior studies have primarily assessed the predictive power of fundamental variables <cit.>, while mostly ignoring textual self-descriptions. Notable exceptions are <cit.> and <cit.>, who use textual self-descriptions for prediction. However, these works rely on traditional methods with manual feature engineering. We thus contribute to the existing literature stream with a novel, fused large language model to combine textual self-descriptions with fundamental variables for predicting startup success.



In this paper, we aim to predict startup success from online profiles of VC platforms. Thereby, we not only consider fundamental information (e.g., on founders, funding, and the business sector) that are captured in traditional scorecards but we also leverage the textual self-descriptions in online profiles on VC platforms. Here, we develop a machine learning approach to predict startup success from large-scale VC platforms. Machine learning allows us to assess how well startup success can be detected for new startups and thus support the decision-making of investors regarding whether to select a startup for funding. Specifically, we develop a tailored, fused machine learning approach for predicting startup success that considers both (structured) fundamental variables and (unstructured) textual self-descriptions. For this, we draw upon large language models as a recent innovation in machine learning <cit.>, which we carefully adapt to our research objective. A key benefit of large language models in practice is that they are pre-trained on a large amount of public data, because of which relatively small datasets are sufficient for fine-tuning and, thus, to generate accurate predictions. We then assess the relative contribution of textual self-descriptions to making predictions of startup success. 



We evaluate our machine learning approach based on our fused large language models for predicting startup success using 20,172 online profiles from Crunchbase. Crunchbase is one of the largest online VC platforms hosting online profiles from startups. We find that only fundamental variables can alone make predictions with a balanced accuracy of 72.00 %. When additionally incorporating textual self-descriptions, the balanced accuracy increases to 74.33 %. The improvement is statistically significant, implying that textual self-descriptions are effective in predicting startup success. In addition, we estimate the financial performance of our machine learning approach by translating the performance improvement to investment portfolio improvement. The investment portfolio improvement amounts to a 40.61 percentage points increase in return on investment (ROI) when incorporating textual self-descriptions, highlighting the practical implications of our machine learning approach. We then evaluate the prediction performance across various events indicating startup success (, initial public offering, acquisition, and external funding). We further provide an extensive series of sensitivity analyses in which we compare the prediction performance across business sectors, startup age, and additional machine learning baselines, thereby confirming the robustness of our findings. 




Our work contributes to business analytics in several ways. First, we provide empirical evidence on the operational value of online VC platforms for better investment decision-making. Thereby, we extend upon extensive research which has studied the benefits of online platforms for users while we focus on investors. Second, we contribute to a growing stream of machine learning in business analytics <cit.>. Here, we demonstrate an impactful application of machine learning in VC decision-making. Third, we show the operational value of large language models for research and practice. However, as we detail later, a naïve application of large language models would miss significant predictive power. Instead, our task requires a non-trivial adaptation through a fused large language model to our decision problem in order to make combined predictions from both fundamental information and texts. Fourth, we provide a flexible tool for investors to automate their screening process in VC decision-making.



The rest of this paper is structured as follows. <Ref> provides a background on venture capital and analytics for decision-making. In <Ref>, we develop our machine learning approach in the form of a tailored large language model. <Ref> presents our dataset with online profiles from Crunchbase, based on which we study the predictive power of textual self-descriptions (<Ref>). We then discuss implications for both business analytics practice and research (<Ref>), while <Ref> concludes.



§ RELATED WORK





 §.§ Venture Capital




Startups are new entrepreneurial ventures founded to develop and validate a business model <cit.>. In practice, startups typically take an innovative idea and then build a scalable business model around it, with the intention of turning the startup into a high-growth, profitable company <cit.>. This process is largely dependent on external funding in order to cover costs for technology development, entering markets, or other upfront investments. Hence, events in which startups receive funding are commonly used in the literature to determine success <cit.>. Examples of such events are initial public offerings (, Airbnb, which went public in December 2020), acquisitions (, Slack and DeepMind, which were acquired by Salesforce and Google, respectively), and external funding (, SpaceX, which had several funding rounds after its series A funding in 2002). To capture startup success, prior literature has often studied either individual events such as initial public offerings <cit.> or a combination of events <cit.>.



Startups often represent lucrative investment opportunities with the prospect of large returns. As of 2023, more than 180 startups have turned into unicorns, that is, reached valuations of over USD 1 billion in less than five years <cit.>. Investing in such unicorns in an early stage can create a return multiple times larger than the initial investment. However, investments in startups are known to be of high risk. Startups that eventually fail leave the investor with little or no return. Hence, identifying successful startups at an early stage is difficult <cit.>.  




Predicting which startups will turn out to be successful is inherently challenging, as startups represent new ventures for which little to no information on past performance is available. Thus, many investors make such predictions based on their gut feeling <cit.>. However, according to prior literature, there are several determinants that characterize successful startups. These can be loosely divided into characteristics regarding the business model, the founders, and funding. (1) The business model explains—to some degree—the survival of ventures <cit.>. In this regard, the business sector is also associated with startup success <cit.>. (2) Founders decide upon how a business is run and thus founder characteristics are important success factors <cit.>. For instance, startups are more likely to be on a path toward growth when their founders have attended higher education <cit.>. 
 (3) Funding is often a prerequisite to stimulate growth <cit.>. Hence, startup success is also associated with previous funding rounds <cit.>. In this regard, it is further beneficial for startups to have backing from a known venture capitalist <cit.>. Hence, to avoid relying on gut feeling or subjective bias when processing information about startups, machine learning presents a scalable, data-driven approach to predict startup success.



 §.§ Predicting Startup Success




Prior works have developed data-driven approaches for predicting startup success. For instance, predictions can be made based on data from questionnaires, namely via so-called scorecards <cit.>. One study also draws upon data that are extracted from business plan competitions <cit.>. Yet, both questionnaires and business plan competitions involve data from manual reporting, which is often not available in VC markets. These data sources also tend to have limited coverage, and thus their usefulness in the daily decision-making of investors is limited. A different stream of literature predicts acquisitions as a specific event in startup lifecycles using proprietary databases (e.g., COMPUSTAT <cit.>, SDC Platinum <cit.>). However, such databases are typically restricted to specific events (, acquisitions) and, on top of that, have limited coverage as they provide only few variables (e.g., about founders and funding) but not textual descriptions. In contrast to that, textual descriptions about startups and their business model, innovation, or market structure may provide significant predictive power regarding which ventures will eventually be successful. 



Recently, the possibility of using online data from VC platforms to predict startup success was explored <cit.>. Predicting startup success from VC platforms has a clear advantage in practice: online platforms for VC typically exhibit comprehensive coverage of startups and thus provide big data <cit.>. This is beneficial, as large-scale datasets are generally a prerequisite for making accurate inferences using machine learning. Studies predicting startup success based on questionnaires have often relied on samples with less than 200 observations (, 200 different startups), because of which the prediction models can not generalize well across startups and thus lack predictive power <cit.>. In contrast, online platforms for VC, such as Crunchbase, provide online profiles of more than 20,000 startups in the U. S alone. 

Based on data from VC platforms, a variety of research questions have been studied. <cit.> evaluate the predictive ability of fundamental information at Crunchbase, but textual self-reports as predictors are ignored. In <cit.>, a hybrid machine learning approach is designed in which both fundamental information and judgment scores from crowds are combined, but again textual self-reports as predictors are again ignored. <cit.> use textual self-reports for prediction but rely on traditional, bag-of-words representations and not large language models. <cit.> also make predictions from self-reports but rely on a dictionary-based approach that requires manual feature engineering. Hence, the ability of large language models together with textual self-descriptions from VC platforms has yet to be explored and presents our contribution.  




 §.§ Machine Learning in Business Analytics




Machine learning can support managerial decision-making by predicting uncertain operational outcomes <cit.>. The adoption of machine learning in business analytics has been greatly fueled by the increasing availability of data and recent methodological advances <cit.>.  
Promising examples include credit scoring <cit.>, financial risk assessment <cit.>, business failure prediction <cit.>, throughput prediction <cit.>, customer analytics <cit.>, recommendation systems <cit.>, and public sector operations <cit.>. However, the aforementioned works build upon structured data and not text. 

Business analytics has also increasingly embraced machine learning that can make inferences from textual content <cit.>. As such, business analytics can mine user-generated content, e.g., from social media, in an automated and scalable manner <cit.>. For example, <cit.> enrich historical sales data with social media as a measure of customer perception towards products and evaluate how that combined data source is better in predicting future sales.  However, existing methods in business analytics oftentimes build upon bag-of-words approaches where an unordered set of words is used as input <cit.> and where, as a result, the relationship, order, and hierarchical structure among words is lost. Hence, existing methods merely operate on word frequencies and not on semantic meaning. A potential remedy is given by large language models that model the ordered sequence of words and thus capture the semantics of running text; however, the operational value of large language models has so far been largely unclear. Moreover, we are not aware of previous work that uses large language models for startup prediction to support investment decisions. 


	


§ EMPIRICAL MODEL



In this section, we first formulate our research question of whether textual self-descriptions from VC platforms predict startup success. To answer this, we then describe our machine learning approach based on a tailored, fused large language model. 



 §.§ Research Question




In this study, we build a machine learning approach where we leverage information provided by startups on VC platforms in order to predict startup success. Information on online VC platforms such as Crunchbase can loosely be grouped into two categories (which may potentially complement each other). (1) VC platforms provide structured information on a startup's fundamentals. Examples of such fundamental variables are the age of the startup, the number of founders, or information about past funding success. Fundamental variables are typically entered on VC platforms in a structured format and thus with little degree of customization. (2) Startups can additionally provide a textual self-description. The textual self-description can be used to describe the business model, a startup's innovation, or the market structure. Textual self-descriptions have become mandatory on VC platforms such as Crunchbase but the actual content is at the startup's discretion. 


In this study, we examine whether large language models can be successfully leveraged by investors to predict startup success from textual self-descriptions on VC. There are several factors that lead us to expect that textual self-descriptions are predictive. In particular, startups can use the textual self-description on VC platforms to present information on a startup's business model, innovation, or market structure. An example is BetterTrainers has a new type of business model that protects all sessions booked through the site with premium insurance coverage where a business model is explained, or FaceTec's patented, industry-leading 3D Face Authentication software anchors digital identity with 3D FaceMaps where a startup details how to make use of certain technologies. Besides the actual information captured in the text, latent factors such as the tone of the text (e.g., a positive sentiment) may also implicitly signal success. 


As seen by the previous examples, traditional approaches from machine learning for making predictions from online descriptions (e.g., bag-of-words) will likely struggle with the complexity of the underlying task since traditional approaches only rely upon word frequencies and do not provide a principled approach to infer semantic meanings. To this end, the previous examples motivate the use of large language models in our study as a principled, data-driven approach to capture semantic meanings in text and thus to predict startup success.




 §.§ Fused Machine Learning Approach


In the following, we present our fused machine learning approach in order to predict startup success. Let i = 1, …, n denote the startups. Specifically, we develop a tailored, fused large language model as shown in <Ref>. In our machine learning approach, both sets of variables – i.e., fundamental variables (FV) and textual self-descriptions (TSD) – are taken into account but in different ways. (1) The fundamental variables come in a structured format x_i^FV∈ℝ^m_FV and are thus directly passed on to the final machine learning classifier. (2) The textual self-descriptions are first mapped onto document embeddings x_i^TSD∈ℝ^m_TSD and then passed on to the final ML classifier. Let us denote the final ML classifier by ϕ_θ : ℝ^m_FV + m_TSD→{ 0, 1} with some parameters θ. Here, the output y_i ∈{ 0, 1 } indicates whether a startup i = 1, … will be successful (y_i=1) or not (y_i=0). Crucially, a custom architecture for our large language model is necessary in order to fuse both fundamental variables x_i^FV and document embeddings x_i^TSD to make predictions. For comparison, we later evaluate a naïve large language model without the “fused” structure which uses only x_i^TSD for prediction.


In our machine learning approach, we take the textual self-description of the startup and use a large language model <cit.> as an embedding generator to map text onto a document embedding. The document embedding is then concatenated with the fundamental variables and the resulting concatenated vector is then used as input to the classifier. Large language models represent state-of-the-art techniques for modeling natural language in machine learning <cit.>. A prominent example is BERT, which has been found effective in capturing complex dependencies such as semantics in textual content <cit.>. In the following, we detail how we fuse data in our large language model. 




 


  §.§.§ Large Language Model (BERT) as embedding generator.




Large language models, often also called transformers, are large-scale deep neural networks that are carefully designed to process running text <cit.>. 
The practical benefit of large language models is that they leverage the strength of large-scale deep neural networks and are thus able to capture context, semantics, structure, and meaning <cit.>.




A prominent large language model is BERT <cit.>. BERT was developed by Google AI and stands for bidirectional encoder representations from transformer. BERT has been successful in solving various machine learning tasks for natural language. 
In particular, BERT has been shown to be superior to alternative document representations such as bag-of-words. Methodologically, 

Language models such as BERT map running text onto a new representation called embedding <cit.>. Formally, each textual input is first transformed into a sequence of tokens [[CLS], w_1, w_2 …] based on the predefined vocabulary of BERT, with [CLS] being used at the beginning of each sequence. Hence, for each textual input, BERT receives a sequence of individual tokens as input where the tokens are represented by vectors [CLS], w_1, w_2, …∈ℝ^w.  The vectors are not “one-hot-encoded” as traditionally done in simpler models. Instead, BERT uses an embedding layer to convert the sequence of tokens into dense vector representations e_[CLS], e_1, e_2, …∈ℝ^e that are lower-dimensional (i.e., the dimensionality e is much smaller than the dimensionality of a typical one-hot encoding, which is computationally more desirable). Next, the token embeddings are fed into a transformer encoder. A transformer encoder is a neural network designed for sequential data that processes the entire input sequence [e_[CLS], e_1, e_2, …] simultaneously, rather than sequentially. It relies on two key mechanisms: (a) positional encodings, which add information about the position of each token to retain the order; and (b) an attention mechanism, which allows the model to weigh the importance of different tokens dynamically. Thereby, a transformer encoder employs a complex, non-linear process to determine how tokens influence one another. The output of the transformer encoder consists of transformed vectors (embeddings) [o_[CLS], o_1, o_2, …], which can then be used for various tasks. Specifically, the embedding for the [CLS] token (i.e., o_[CLS]) can be used for classification tasks as it aggregates the meaning of the entire input sequence. 

During training, BERT utilizes a technique called masked language modeling, where some of the input tokens are randomly masked (i.e., omitted) for self-supervised learning. The objective of BERT during training is to correctly predict these masked tokens. Thereby, BERT updates its internal weights and learns a deep understanding of language context and relationships between words. Due to self-supervised learning, large-scale textual databases (e.g., Wikipedia) can be used for training but without the need for explicitly annotated labels. A schematic visualization is in <Ref>.





Our implementation is as follows. We use the so-called basic, uncased version of BERT <cit.>, comprised of 12 layers with ∼ 110 million trainable parameters. It generates embeddings o_[CLS], o_1, o_2, …∈ℝ^n of dimension n = 768. BERT is shipped as a pre-trained network where parameters have already been learned from open-source content. Before applying BERT, all text is lowercased and tokenized using the WordPiece algorithm, which maps the text onto subwords or unigrams from the WordPiece vocabulary. Afterward, the text is passed through the pre-trained BERT network. The embedding of the [CLS] token (i.e., o_[CLS]) is then used as the document embedding x^TSD, representing the textual self-description for the downstream classification. Hence, our document embedding x^TSD is of dimension m_TSD=786.





  §.§.§ Baseline text representations.



We compare our machine learning approach based on a tailored, fused large language model against three traditional text representations. All of the baselines are again concatenated to the fundamental variables and are then fed into a final machine learning classifier.  The final machine learning classifier is again subject to rigorous hyperparameter tuning (see later for details) for fair comparison. Therefore, all performance gains from our approach must be attributed to that large language models are better at handling textual content. 



  * Manual feature extraction: The first text-based baseline is based on manual feature extraction. Specifically, we manually craft features that capture textual information (e.g., the length, the mean word length, and the number of geographic references). We follow prior literature and extract the same features as in <cit.>. This results in a text representation of dimension 10. We refer readers to <cit.> for a full list of the features.


  * Bag-of-words:  We compare our machine learning approach against the traditional approach of a bag-of-words baseline. We refer readers to <cit.> for an introduction. We implement bag-of-words as follows. We first tokenize the words of the textual self-description to unigrams, remove stop words, lemmatize, and apply a tf-idf weighting. Furthermore, we remove words with more than 95 % sparsity. The bag-of-words baseline results in a 98-dimensional text representation.


  * GloVe:  GloVe  <cit.> transforms words into vectors (so-called word embeddings) based on their co-occurrence in a text corpus. Thereby, the vectors capture semantic relationships, offering a rich set of features for text analysis. We use the GloVe model pre-trained on Wikipedia (i.e., ) to extract the 50-dimensional word embeddings. We average the individual word embeddings to get the final text representation used for the downstream classification.






  §.§.§ Final Machine Learning Classifier.



The final machine learning classifier ϕ_θ(·) with parameters θ is responsible for the “fused” approach and, for this, receives the concatenated vector of (1) fundamental variables and (2) document embeddings. The output is then the predicted probability of startup success.  
We thus optimize 

    θ^∗ = _θ𝔼[ ℒ(ϕ_θ([x^FV, x^TSV]), y) ] ,

where ℒ is a convex loss (e.g., mean squared error) and where [ · , ·] is the concatenation operator. 



We experiment with different classifiers that are designed to handle both linear and non-linear relationships in the data. Specifically, we make use of the following classifiers:



  * Logistic regression: The logistic regression is a simple linear model used for binary classification. It models the probability of a binary outcome using the logit function to map predictions to probabilities. The logistic model expresses the log-odds of the outcome variable as a linear combination of the independent variables, formalized as log(p/1-p)=θ^T x, where p is the probability of the outcome of interest.

  * Elastic net: The elastic net extends the logistic regression in which overfitting is prevented through regularization <cit.>. Specifically, regularization is given by a combination of both an L1- and an L2-norm penalty (analogous to lasso and ridge methods, respectively). This thus shrinks some coefficients closer to zero, and, as a result, the classifier generalizes better to out-of-sample observations. Formally, let ϕ_θ(x) = θ^T x. Then the regularized loss ℒ_reg is formalized as ℒ_reg(x,y) = ℒ(ϕ_θ(x), y) + λ ( 1-α/2θ^2_2 + α θ_1 ) with hyperparameters α and λ. The elastic net is especially beneficial in tasks where predictors are subject to linear dependence <cit.>. For reasons of completeness, we also experimented with lasso and ridge methods <cit.>, but with qualitatively similar results (and thus omitted the results for brevity). 

  * Random forest: The random forest is an ensemble learning classifier where predictions are made from a multitude of decision trees <cit.>. Each decision tree is fit to a random subset of the data, while the final prediction is then made by taking the majority vote over the individual decision trees. As a result, the classifier is less prone to overfitting, has a better prediction performance than a single decision tree, and is effective in handling non-linear relationships <cit.>. 

  * Neural net: The neural network is a flexible model for classification by using layers of nodes that transform the input through non-linear activation functions. The output layer uses a sigmoid to produce class probabilities. The loss is regularized by a combination of both the L1- and L2-norm penalty to prevent overfitting. Neural networks excel due to their flexibility in handling non-linear
relationships.






 §.§ Performance Metrics


To evaluate the performance of machine learning in predicting startup success, we report different performance metrics: balanced accuracy, precision, recall, F_1-score, area under the curve from the precision-recall curve (AUCPR), and area under the curve from the receiver operating characteristics (AUROC). However, due to its inherent benefit of considering the complete distribution of discrimination thresholds <cit.>, we primarily focus on the AUROC. We remind that we follow common practice in machine learning and evaluate the performance on out-of-sample observations, that is, startups that have not been part of the training set but from the test set so that they are thus unseen to the machine learning classifiers. 




Furthermore, we calculate the return on investment (ROI) for the machine learning-selected portfolios. Let 𝑇𝑃 denote the number of true positives (, cases where startup success was predicted correctly) and 𝐹𝑃 the number of false positives (, cases where the model predicted success despite that the startup was actually not successful). We then calculate the net investment gain for correctly predicted successful startups by taking the sum of the final investment values 𝐹𝐼𝑉_𝑇𝑃 (, the startup valuations after a success event) minus the sum of the total cost of investment (IC).
Note that, since data on startup valuations and costs of investment is not always publicly available, we approximate these variables using constants determined based on historical mean values for startups listed on Crunchbase.[In our Crunchbase dataset (see <Ref>), the valuation of a startup after a success event (, initial public offerings, funding, acquisitions) is, on average, $184.47 million. The pre-success valuation (, the last valuation in previous funding rounds) is, on average, $12.19 million. Hence, startups have, on average, a 15.13 times higher valuation if they become successful.]
For companies that were non-successful, 
we conservatively assign a final investment value of zero (𝐹𝐼𝑉_𝐹𝑃). The ROI for the portfolio is then calculated by taking the net investment gain divided by the total cost of investment. Formally,

    ROI = 𝑇𝑃×𝐹𝐼𝑉_𝑇𝑃 + 𝐹𝑃×𝐹𝐼𝑉_𝐹𝑃 - (𝑇𝑃 + 𝐹𝑃) ×𝐼𝐶/(𝑇𝑃 + 𝐹𝑃) ×𝐼𝐶× 100,

where the total investment costs (𝐼𝐶) comprise (i) the investor's investment into equity of the startup; and 
(ii) we consider 10% of the last valuation as additional screening and monitoring costs for the investor.

   


 §.§ Implementation Details




Our implementation follows best practice in machine learning <cit.>. For this, we split the data into a training set and a test set. The former is used for training the model; the latter is used to evaluate the out-of-sample performance. In our work, we randomly assign 80 % of the data samples to the training set and 20 % to the test set. Due to class imbalances, common procedures in machine learning are followed; that is, we apply a stratified split <cit.>, so that both sets have the same ratio of successful vs. non-successful startups. To ensure robustness in our evaluation, we repeat the random split five times and report the mean and standard deviation of the performance metrics on the test set across the five iterations. This allows us to quantify how well machine learning can predict success for ventures that were not seen during training. 

Hyperparameter tuning is conducted using 10-fold cross-validation. Specifically, hyperparameters are tuned via randomized grid search (20 iterations), using the tuning grid in <Ref>. The best hyperparameters are selected based on the cross-validated AUROC score. Note that the hyperparameter tuning is done separately for the different input variables, that is, for when training our machine learning approach using fundamental variables (FV), textual self-descriptions (TSD), or a combination of both (FV + TSD).






§ EMPIRICAL SETTING






 §.§ Online Profiles on Crunchbase




Our evaluations are based on data from Crunchbase.[<http://www.crunchbase.com>] Crunchbase is a leading online VC platform that connects startups and investors. For this, Crunchbase allows startups to create online profiles where they can present information on their business, founders, and funding. Edits can be made by verified employees to ensure that correct information is entered.


We collected online profiles (, both fundamental variables and textual self-descriptions) from all US-based startups that were listed on Crunchbase. Furthermore, we excluded startups that went public and that have already received series C funding (or a later funding round). The latter is important as our objective is to make predictions for companies that fall under the definition of a startup.




 §.§ Definition of Startup Success


In our study, we predict startup success with regard to different events that are conventionally used as indicators of success <cit.>, namely, whether startups had an initial public offering, have been acquired, or secured external funding. If any of these events occurred, we treat the startup as successful. Otherwise, a startup is treated as non-successful. If not stated otherwise, these labels are used to evaluate our machine learning approach. As part of our sensitivity analyses, we later continue to compare how the prediction performance varies across these events – , initial public offerings, funding, and acquisitions.



 §.§ Time-Aware Prediction and Evaluation Framework



We implemented a time-aware approach that is common in time-series forecasting <cit.>. Recall that we aim to evaluate whether we can predict if startups will become successful in the future. Consequently, we processed our data as follows. We restricted our analysis to startups that were founded between 2013 and 2015, based on which we predicted their future development until the end of 2020. We obtained raw access to the Crunchbase database with historical data. This allowed us to collect information from online profiles that were available in 2015. In particular, we discarded information that was added or updated later, so that we only considered data as presented on Crunchbase at the end of 2015. 

We then predict whether an event indicating startup success has occurred during the years 2016 through 2020, that is, we make forecasts whether startups were successful over a time horizon of five years. The forecast horizon is set analogous to earlier statistics reporting upon a high failure rate among startups in their early stage <cit.>, so that a 5-year-ahead forecast horizon should be sufficient to distinguish successful from non-successful startups. Our choice of events representing startup success is listed in the previous section. 




 §.§ Variable Descriptions





Our fused machine learning approach makes use of an extensive set of variables from Crunchbase (see <Ref>). The outcome variable (, the variable to predict) is binary, denoting whether a startup was successful (=1; otherwise =0). 






The predictors (, the variables that are fed into our machine learning classifiers) consist of the following: (structured) fundamental variables and (unstructured) textual self-descriptions. (1) The fundamental variables (FV) describe different characteristics of startups such as their age or the industries in which they operate (see <Ref>). Note that we use the industries as reported on Crunchbase, which is based on a highly granular scheme (e.g., an Internet-of-Things company may be assigned simultaneously to “artificial intelligence”, “industrial automation”, etc.). Social media activity has been found to be related to startup success <cit.>, and, analogously, we include information about whether startups are on social media (, whether they have a Twitter/X or LinkedIn profile). Furthermore, we collect information about the characteristics of the founders (, the number of university degrees). We follow previous literature <cit.> by controlling for the presence of known investors that have a profile on Crunchbase themselves. We also include information on previous funding rounds but, since we use a historical view on Crunchbase data, we only access information up to our time point when making the predictions so that there is no lookahead bias (i.e., we discard funding rounds that occur during the forecast horizon to ensure a time-aware evaluation framework).[We also considered a less sparse encoding of business sectors (rather than fine-grained industries as in the Crunchbase coding scheme) but we discarded this. The reason is seen in our later analysis, where there is little variability across business sectors and, thus, sector information has only little predictive power. Furthermore, we also considered additional information about founders (e.g., their number of current and past jobs) and prominence (e.g., site visitors, growth in site visitors, number of media articles) but found that these are too sparse to make a meaningful addition to our predictions.] (2) The latter, , textual self-descriptions (TSD), are encoded via the large language model (BERT). This yields document embeddings, which are then used as input to the machine learning classifier. 





 §.§ Descriptive Statistics



The above filtering yields a final dataset with 20172 startups. Descriptive statistics on startups for our dataset are as follows (see <Ref>). Out of all startups, 7252 (, 35.94 %) startups have been labeled as successful, whereas 12920 (, 64.06 %) have been labeled as being non-successful. Frequent events indicating success are founding rounds (, 32.45 % of all startups), followed by acquisitions (3.10 %) and initial public offerings (0.40 %). For startups in our dataset, the average age is 18 months. Startups tend to be more successful if they provide a link to their social media profiles. In general, startups are more frequently founded by males (, 1.58 male founders per startup) than by females (, 0.25 female founders per startup). Successful startups have, on average, more founders (mean: 1.98) than non-successful ones (mean: 1.69). Furthermore, founders with university degrees often have more successful startups. On average, startups have previously raised funding totaling to USD 3.156 million from 2.07 investors. Unsurprisingly, startups that are eventually labeled as successful have received more funding (mean: USD 4.81 million) and are backed by more investors (mean: 3.28). On average, successful startups provide a shorter textual self-description (mean: 613.32 characters) than non-successful ones (mean: 694.04 characters). <Ref> lists two example textual self-descriptions, one for each class.








Startups listed on Crunchbase operate in a variety of business sectors (see <Ref>). The majority of startups in our data operate in the area of Information Technology and Communication Services. In contrast, startups in the Energy, Utilities, and Materials sectors are less common. Note that startups can be assigned to multiple business sectors. Across the business sectors, we also see variation in the success rate of startups. For instance, startups in some sectors such as Utilities have a high success rate (50.79 %), while the success rate for Communication Services amounts to only 32.71 %. 







§ EMPIRICAL FINDINGS






 §.§ Comparison of Our Large Language Model Against the Baselines


We now evaluate the performance of our fused large language model in predicting startup success (see <Ref>). We use the neural net as the best-performing final machine learning classifier within our fused large language model for this evaluation. For a detailed comparison across different final machine learning classifiers, we refer to <Ref> of the Supplementary Materials. We further remind that we follow common practice in machine learning and evaluate the performance on out-of-sample observations, that is, startups that have not been part of the training set and are thus unseen to the machine learning classifiers. In addition, we repeat the random splitting of our train and test sets five times and thus report the mean and standard deviation of our evaluation metrics across the five test sets.[We also perform an out-of-time evaluation in <Ref> of the Supplementary Materials, where we evaluate the performance in predicting the success of startups that originate from a period outside the one used for training. Overall, the performance remains robust but, due to the task formalization, has a smaller sample size and thus tends to have a larger variance.]

Overall, we find that our tailored, fused large languages model is considerably more accurate than a majority vote (i.e., a model that always predicts the majority class label) and a random vote (i.e., a model that predicts class labels randomly based on the distribution of the class labels in the training data). Both approaches represent naïve baselines from machine learning, which are outperformed by a large margin. Our tailored, fused large language model using both fundamentals and textual self-descriptions yields an AUROC of 82.78 %, a balanced accuracy of 74.33 %, and a 7.23-fold ROI. Altogether, this demonstrates the efficacy of machine learning based on our fused large language model in predicting startup success from VC platforms.

We further compare our fused large language model against common baseline text representations. Specifically, we draw upon manual feature extraction from textual data <cit.>, GloVe document embeddings <cit.>, and a bag-of-words approach <cit.>. The baseline text representations have a known limitation in that they struggle with capturing long-term dependencies across language, because of which semantics are ignored to a large extent. As expected, we find that, compared to our fused large language model, the baselines are inferior. For example, the best baseline in terms of AUROC (GloVe) has a 6.41-fold ROI, while our custom, fused large language model has a 7.23-fold ROI, which is a plus of 82.19 percentage points. Note that both our fused, large language model and the bag-of-words baseline have access to the same data, that is, fundamental variables and textual self-descriptions. Hence, all performance improvements must solely be attributed to the better model architecture of our fused large language model.




In addition, we compare using fundamental variables only vs. a combination of fundamental variables and the textual self-description. Here, including textual self-description using the baseline text representations increases the AUROC by 0.62 percentage points (manual feature extraction <cit.>), 1.29 percentage points (GloVe <cit.>), and 0.5 percentage points (bag-of-words). Including textual self-descriptions within our fused large language model performs best and increases the AUROC by 2.18 percentage points. As such, we yield consistent evidence that demonstrates the operational value of textual self-descriptions: a significant improvement in prediction performance is achieved by including textual self-descriptions. Altogether, this highlights the importance of textual self-descriptions for successful investing decisions.




 §.§ Sensitivity to Final Machine Learning Classifier



We now provide a sensitivity analysis where we vary the final machine learning classifier (i.e., elastic net, random forest, neural network) and  within our fused large language model.[We also tested the performance of varying the input variables (i.e., FV, TSD, FV + TSD) within our final machine learning classifiers. We report the results in <Ref>.] Thereby, we confirm that our choice of a neural network for the final machine learning classifier in our fused large language model is superior. The results are reported in <Ref>. By varying the final machine learning classifier in our fused large language model using both fundamentals and textual self-descriptions, we yield an AUROC of 81.76 % (logistic regression), 82.51 % (elastic net), 81.75 % (random forest), and 82.78 % (neural network). We observe a similar pattern with regard to the other performance metrics. For instance, the neural network achieves a 7.23-fold ROI. Hence, the best overall AUROC is obtained by the neural network, followed by the elastic net, logistic regression, and the random forest. Altogether, this demonstrates the efficacy of our fused large language model based on a neural network in predicting startup success from VC platforms.


  





 §.§ Sensitivity to Fine-Tuning Our Large Language Model



We now experiment with fine-tuning our fused large language model. Specifically, we add a classification head to the [CLS] embedding (i.e., o_[CLS]) for classifying startup success on top of BERT. We concatenate the fundamental variables to the [CLS] embedding before feeding them to the classification head. This way, both the classification head is trained and BERT is fine-tuned simultaneously based on the task of predicting startup success. Hence, the difference to no fine-tuning lies in the fact that we now allow for parameters in BERT to be fine-tuned for the task of classification.

We fine-tuned BERT using the transformers framework from Huggingface <cit.>. We use a training batch size of 32 and a learning rate of 4 · 10^-5. We freeze the first 8 layers as they capture language patterns and an understanding of text in general. We update the weights of BERT and the classification head using the AdamW optimizer <cit.>. We fine-tune for a maximum number of 5 epochs. We validate the performance every 50 steps. We performed early stopping when the loss on the hold-out set does not decrease for more than 5 steps.

<Ref> reports the results. Overall, we do not observe any performance improvement when fine-tuning our fused-large language model. The performance of fine-tuning is comparable to that of our fused large language model with a neural net classifier. Specifically, fine-tuning our fused large language model yields a 0.12 percentage point decrease in accuracy, 0.1 percentage point decrease in AUROC, and 10.64 percentage point decrease in ROI, as compared to our not fine-tuned fused large language model. Hence, the pre-trained embeddings of BERT already capture textual information relevant to the task of success prediction. Our findings underline an important aspect of machine learning: Increasing the number of trainable (or fine-tunable) parameters does not necessarily guarantee performance improvements. We discuss the finding later in <Ref>.









 §.§ Prediction Performance Across Business Sectors




We now perform a sensitivity analysis in which we compare how the prediction performance from our fused large language model varies across business sectors (see <Ref>). In general, startup activities and outcomes vary significantly across business sectors <cit.>. For example, the sector of Information Technology typically features better data coverage and a higher number of startups, potentially leading to better predictability. Motivated by these differences, we perform a sensitivity analysis to provide insights into the extent to which textual self-descriptions contribute to performance gains across sectors. Here, we focus our evaluations on the implementation based on a neural network, , the best-performing classifier. We compare high-level business sectors for easier interpretability (this is different from the fine-grained but sparse industries that are reported on Crunchbase and that we use as predictors). Overall, we find that the prediction performance is fairly robust. The AUROC varies from 72.04 % (Energy) to 85.39 % (Industrials). This thus confirms that our fused large language model allows for accurate predictions across all business sectors. Furthermore, including textual self-descriptions improves the prediction performance across most business sectors. The only exceptions are the four sectors with the smallest number of data points (Energy, Materials, Real Estate, and Utilities). For these sectors, including textual self-descriptions does not lead to a performance improvement as compared to using only the fundamental variables for prediction. Also, the standard deviation in the prediction performance is higher across these sectors. This implies that a sufficient number of training observations is necessary to make accurate predictions from textual self-descriptions.[We also tested whether the lower prediction performance in these business sectors could stem from more diverse textual self-descriptions as compared to other business sectors. However, the representations of the textual self-descriptions are (a) not more/less discriminatory for successful vs. non-successful startups across business sectors, and (b) not more/less diverse across business sectors, suggesting that more diversity in self-descriptions within specific business sectors is not a factor for lower prediction performance.]











 §.§ Prediction Performance Across Investment Events


We compare the prediction performance of our fused large language model across different events that are indicative of startup success, namely initial public offering, acquisition, and external funding. For this, we evaluate our models on subsets of the out-of-sample test sets split by the different events. Hence, the corresponding accuracy quantifies, for example, to what extent startups are correctly classified in the subset of startups that eventually had an initial public offering. We proceed analogously for acquisition and funding events. The results are reported in <Ref>. Overall, the events vary in their frequency, as only a few startups had an initial public offering or had been acquired, whereas a larger proportion received external funding. 


We find that the prediction performance is generally higher for initial public offerings and funding events. Here 82.05 % of initial public offerings and 80.17 % of funding events were predicted correctly. In contrast, only 65.54 % of acquisitions were predicted correctly, implying that, for the latter, inferences are more challenging. Again, we confirm that machine learning benefits from incorporating textual self-descriptions. In fact, using textual self-descriptions increases the rate of correct classifications for initial public offerings by 4.02 %, for acquisitions by 3.92 %, and for funding events by 2.75 %. Therefore, our findings suggest that textual self-descriptions from VC platforms are informative for predicting startup success, consistently across all success events.







 §.§ Robustness Checks



We perform the following additional robustness checks. We evaluate the prediction performance of our fused large language model across different company characteristics (, the age of a startup) and the length of the textual self-description. We find that the inclusion of textual self-descriptions improves the prediction performance considerably, which is consistent across startup ages and across different text lengths. This contributes to the robustness of our findings.




  §.§.§ Prediction Performance Across Startup Age.



The prediction performance with and without textual self-descriptions grouped across startup age is reported in <Ref>. For all age groups, the majority vote and random vote as naïve baselines from machine learning are outperformed by a considerable margin and thus point toward the overall large prediction performance. In addition, all performance metrics increase by a considerable margin when including textual self-descriptions. This adds further robustness to our finding that textual self-descriptions are predictive of startup success. Furthermore, the balanced accuracy is higher for older startups with and without textual self-descriptions, indicating that more established startups potentially yield more predictive information on Crunchbase.





Varying the age of startups is also important for another reason: it allows us to assess the prediction performance across different time periods. Startups with an age between 1–12 months originate from 2015, an age between 13–24 months originate from 2014, etc. This thus contributes to the robustness of our findings.




  §.§.§ Prediction Performance Across Length of Textual Self-Description.



The prediction performance with and without textual self-descriptions across different lengths of the textual self-description is reported in <Ref>. For all length groups, a majority vote and a random vote are outperformed by machine learning. In addition, a clear improvement in AUROC is found for all groups when including textual self-descriptions. Overall, this adds robustness to our finding that textual self-descriptions are predictive of startup success. Furthermore, the AUROC is higher for startups with longer textual self-descriptions. Similarly, both metrics increase for the baseline without textual self-descriptions. Still, the length of the textual self-description appears to play a minor role in the prediction performance.







 §.§ Post-Hoc Explainability of Our Machine Learning Approach

 
 

The previous analyses demonstrate the performance improvement of including textual self-descriptions for the task of startup success prediction. Now, we analyze the contributions of each variable (, fundamentals and textual self-descriptions) for predicting startup success. To this end, we aim to understand how our fused large language model uses the variables to arrive at predictions. We use the SHAP value method <cit.>. Intuitively, the SHAP value method treats the prediction of a model as a cooperative game, i.e., the prediction (i.e., the payoff) must be allocated fairly among the feature values (i.e., the individual players) based on their contribution. Hence, the SHAP value method enables a nuanced understanding of how each feature contributes to the prediction of the model and is frequently used for understanding machine learning in management applications <cit.>.

SHAP values are computed for each observation separately, i.e., every feature within the vector of each observation is assigned a SHAP value. SHAP values can also be interpreted at the model level. Therefore, we quantify both feature attribution and feature importance based on the SHAP values. Feature attribution is directly determined by the SHAP values and feature importance is computed by averaging the absolute SHAP values across observations. We follow previous research <cit.> and aggregate the SHAP values (sum) across the document embedding of the TSD to one feature representation.

<Ref> shows the summary plot of SHAP values computed for the predictions of our fused large language model. In the left plot, the dots across each feature represent the feature attribution for each prediction of a specific observation. The right plot shows the mean of the absolute SHAP values across all samples. Both plots show the 20 features with the highest computed feature importance, ranking them from highest (top) to lowest (bottom) importance. Notably, the aggregated representation of the textual self-description is the most important feature, indicating that it contributes, on average, the most to the prediction of our fused large language model. Here feature attributions range from -0.53 to 0.76 with a mean absolute value of 0.29. Thus, out of all features, the textual self-description adds the most to the predictions of our fused large language model. 



Variables characterizing the momentum of a startup also make important contributions to the predictions of our fused large language model. Overall, the age of the startup is the second most important feature. Feature attributions range from -0.28 for a startup age of 35.59 months to 0.20 for a startup age of 4 months. On average, a higher age of startups is estimated to have a negative feature attribution to success prediction. This may indicate that startups with a longer market presence face reduced probabilities of success due to, for example, lower perceived growth potential and questionable viability of their business model if they have not yet achieved success. In addition, recent funding activities (last round time lag) are estimated as positive contributions to predictions of success, with feature attributions ranging from -0.37 to 0.05. 
A plausible explanation might be that recent funding signals reduced risk, as other investors have recently found the startup promising enough to invest in (, a form of validation). Similarly, the number of investors in the last investment round contributes positively to the success predictions, reinforcing the idea that previous funding may serve as a form of validation that predicts success also in the future.

Founder characteristics also play an important role in the predictions made by our fused large language model. Among these, the number of founders and their educational backgrounds are highly influential. Here, the total number of founders positively contributes to success predictions, which suggests that startups with a higher number of founders may benefit from diverse skill sets and shared responsibilities. The total number of degrees among the the founders also shows a positive contribution, with attributions ranging from -0.09 to 0.09. This suggests that a higher number of educational degrees within the founding team may predict success, possibly reflecting the founding team's capability to tackle complex challenges and innovate. In addition, the presence of a LinkedIn profile (and an email) for founders also stands out as an important and positive contributor to the predictions of our model. This indicates that visible professional networking and the credibility it brings might be a strong predictor of later startup success. 

The contributions of the sectors in which a startup operates are also reflected in the SHAP values. For example, as seen by the SHAP values, software is the most influential sector feature. Startups in the software sector typically exhibit high growth potential, so that the information of whether a startup operates in this sector helps to predict later success. Similarly, sectors such as healthcare and artificial intelligence also show positive contributions, with healthcare ranging from -0.04 to 0.13 and artificial intelligence from -0.04 to 0.22. These sectors are typically characterized by innovative solutions of high impact. In contrast, sectors such as food and beverage and hardware show smaller but still positive contributions, which could be attributed to higher capital requirements and longer time to market.











§ DISCUSSION





 §.§ Managerial Implications



Our work demonstrates that VC platforms can be used to predict startup success and thus support investing decisions. We find that predictions from our fused large language model achieve an AUROC of up to 82.78 %, a balanced accuracy of up to 74.33 %, and a 7.33-fold ROI. Thereby, baselines without machine learning (, a majority vote) are outperformed by a considerable margin. Prior literature has already shown that various fundamental variables are predictors of startup success, whereas we show that additional predictive power is offered by textual self-descriptions. Here, we find that incorporating textual self-descriptions through our fused large language model increases the AUROC by 2.18 percentage points, the balanced accuracy by 2.33 percentage points, and the ROI by 52.25 percentage points. The increase in prediction performance is statistically significant. As such, our work is of direct managerial relevance as it provides computerized decision support for venture capitalists with the prospect of making financially rewarding investments. 

We also show that traditional machine learning methods for making predictions from text (e.g., bag-of-words with manual feature extraction <cit.>) are inferior to state-of-the-art methods based on large language models. Traditional methods <cit.> rely on manually crafting features from text that might not capture the entire latent textual information. In contrast, our fused large language model utilizes so-called neural representation learning, capturing latent information in texts through an automated, data-driven procedure that learns from data. Notably, we observe that fine-tuning the language model does not increase the performance. The complexity behind the alignment of pre-trained knowledge and target domain characteristics has been discussed in recent NLP literature <cit.>, where evidence is provided that fine-tuning does not always help performance due to it being highly task- and data-specific. The fact that fine-tuning shows similar performance as no fine-tuning underlines an important aspect of machine learning: increasing the number of trainable or fine-tunable parameters does not necessarily guarantee performance improvements. Thus managers should carefully consider the use of large language models when dealing with decision problems that involve text data. 

The improvements in prediction performance when incorporating textual self-descriptions are robust across all business sectors and economically significant. To assess the practical implications, we translate the prediction performance into investment portfolio performance (ROI). Our results show significantly increased ROI when incorporating textual self-descriptions through our fused large language model: The best-performing baseline without textual self-descriptions amounts to a 6.71-fold ROI, while our fused large language model achieves a 7.23-fold ROI. The financial gains from our fused large language model can be further explained by the substantial costs of false positives in the context of startup investment decisions. False positive classifications for investment decisions lead to investing in startups predicted to succeed but ultimately failing. Hence, investments in startups that eventually fail lead to a potential loss of the entire investment amount. Our model significantly reduces the probability of false positives compared to the baselines, thereby increasing the overall returns from our machine learning approach for making investment decisions.

The Crunchbase database is widely used for academic research, which in turn yields practical implications. Crunchbase offers an online platform with comprehensive data on startups including fundamental variables (e.g., the age of the startup) and textual self-descriptions. Such data has key differences from the data traditionally collected by VC investors for decision-making <cit.>. Here, two reasons stand out why investors traditionally have only little data about startup trajectories. On the one hand, investors typically collect only a few variables about startups (e.g., via scorecards) <cit.> and often not in a structured format <cit.>. On the other hand, and more importantly, investors typically screen only a few dozen startups and thus only have access to startup data for a very small sample size <cit.>, which precludes data-driven inferences. In sum, both of the aforementioned reasons are salient hurdles for training and deploying machine learning tools. As a remedy, prior literature evaluated the predictive ability of fundamental variables on Crunchbase <cit.>. We add to prior literature by using large language models to incorporate the additional predictive ability of textual self-descriptions on Crunchbase. Hence, Crunchbase offers valuable data for VC investors and other practitioners regarding the evaluation of startups and the enhancement of decision-making tools.




 §.§ Methodological Implications


We contribute to business analytics research by demonstrating the operational value of large language models in the context of more effective investment decisions. Thereby, we connect to a growing stream of machine learning in business analytics <cit.>. Different from explanatory analysis (, regression analysis) that merely estimates associations in an in-sample setting, machine learning is concerned with how well inferences can be made in an out-of-sample setting. Here, we demonstrate an impactful application of machine learning in VC decision-making.   



Large language models have several favorable advantages over traditional methods for natural language processing. On the one hand, large language models provide a flexible way to capture semantics and structure in textual materials, thereby bolstering the prediction performance over alternative machine learning approaches (e.g., bag-of-words). On the other hand, large language models can learn from vast amounts of unlabeled texts through pre-training. As such, large language models can often be applied out-of-the-box with little need for fine-tuning. This is beneficial as it greatly reduces the manual effort and the cost for data annotation. However, applications of large language models in business analytics are still rare, while we develop a tailored, fused architecture for our decision-making problem. As shown above, large language models may need custom tailoring. In our case, we build a fused large language model that can leverage running text but where the final prediction layer can also process structured data. As such, we expect that our fused large language model is of direct relevance for many business analytics settings where the goal is to expand traditional operational information in structured form with additional text data. 




Our study offers implications for the use of large language models in business analytics. We based our predictions on a tailored large language model, a recent innovation from machine learning research. We expect that large language models are beneficial for a wide array of managerial decision-making tasks. This opens new opportunities for research by adapting large language models to, for instance, sales and demand forecasting from social media data, credit scoring, and business failure prediction. 



 §.§ Limitations and Future Research



As with other works, ours is not free of limitation, which offers promising directions for future research. First, large language models such as BERT may embed biases that are populated in downstream tasks. Large language models are trained on vast corpora of text data, which inevitably contain societal biases <cit.>. Consequently, there is a risk that these embedded biases could influence predictions <cit.>, potentially disadvantaging certain startups. Addressing this challenge requires ongoing efforts to mitigate biases within large language models. Future research could focus on refining these models to ensure equitable decision-making processes. For now, we call for careful use when deploying our model in practice. Second, our work is centered on data from the VC platform Crunchbase. While this choice is informed by prior research <cit.>, it does introduce a limitation to our work. Crunchbase is a leading online VC platform that collects rich startup and investor data; however, it may not capture the full set of startups and investors globally. Future work might expand the data sources to include a broader spectrum of startups, enhancing the relevance and robustness across different sectors and regions. Third, the economic landscape of startups is dynamically evolving. To ensure ongoing predictive performance, continuous data collection and model retraining is needed. Lastly, the dynamic nature of the economic landscape might lead to startups adapting their textual self-descriptions in response to model predictions. This suggests an area for future research on the equilibrium implications of textual self-descriptions and model predictions. Specifically, analyzing equilibria could unveil the response of startups to prediction models in designing self-descriptions. Such analysis would require a different form of analysis using equilibria but not machine learning as in our paper.



§ CONCLUSION



The majority of startups fail. Owing to this, the decision-making of investors is confronted with considerable challenges in identifying which startups will turn out to be successful. To support investors in this task, we developed a tailored, fused large language model that incorporates the textual self-description of startups alongside other fundamental variables to predict startup success. Here, we show that additional predictive power is offered by the textual self-descriptions. Our model helps investors identify investment targets that promise financial returns. For this, our work provides computerized decision support that allows investors to automate their screening process with data-driven technologies. Furthermore, our study highlights the potential of applying large language models in domains where relevant text data is available but has not traditionally been used for predicting outcomes needed for decision-making. For example, similar to ours, future work could attempt using textual self-descriptions of venture capitalists to predict the performance of their investments. In such scenarios, the findings from our study suggest that combining textual information with conventional data sources may have the potential to significantly enhance predictive accuracy and decision-making processes.




Acknowledgments. We thank Matthias Gey for help.

  
	

informs2014 



 

 





  




Online Supplements












§ OUT-OF-TIME PERFORMANCE EVALUATION


We now repeat the analysis from <Ref> but now perform an out-of-time performance evaluation. Specifically, we split our data, ensuring that startups included in the test set originate from a period that follows the one represented by startups in the training set. Thereby, we can evaluate the ability of our model to predict the success of startups that stem from a period outside the one used for training. We proceed analogously as for the analysis in <Ref>, i.e., we perform a 10-fold cross validation and tune our hyperparameters via a randomized grid search (20 iterations) using the tuning grid from <Ref>. <Ref> lists the results. Overall, our results remain robust, i.e., our fused large language model outperforms all baselines. Our fused large language model using both fundamentals and textual self-descriptions yields an AUROC of 78.91 %, a balanced accuracy of 71.03 %, and a 8.58-fold ROI. However, this type of out-of-time splitting leads to smaller datasets, which increases the variance. Altogether, this demonstrates the efficacy of our fused large language model in predicting the success of startups that stem from a period outside the one represented in the training data.






§ PREDICTION PERFORMANCE OF FINAL MACHINE LEARNING CLASSIFIER FOR THE BASELINES



We now evaluate the performance of the final machine learning classifiers within our baselines in predicting startup success (see <Ref>). Overall we observe some variation as to which final machine learning classifier performs best for each baseline. Specifically, using only the fundamental variables or incorporating the bag-of-words approach the random forest classifier performs best (AUROC: 80.95 or 81.34). For all other baselines, the neural network consistently outperforms the other final machine learning classifiers. We attribute the variation in the best-performing final machine learning classifier to the fact that the input size and the complexity vary for each baseline.





§ PERFORMANCE OF VARYING THE INPUT VARIABLES WITHIN OUR FUSED LARGE LANGUAGE MODEL



<Ref> compares the performance of the final machine learning machine learning classifiers within our fused large language model for varying input variables. Specifically, we assess the relative gain from using textual self-descriptions. For this purpose, we compare the prediction performance with two specific sets of predictors: (a) our machine learning approach trained only on fundamental variables and (b) our machine learning approach trained on both fundamental variables and textual self-descriptions (= our fused large language model). Across all machine learning classifiers, we find consistent evidence that the prediction performance is improved when considering textual self-descriptions. By including textual self-descriptions, the AUROC improves by 2.29 percentage points (logistic regression), 3.01 percentage points (elastic net), 1.8 percentage points (random forest), and 2.18 percentage points (neural network). The improvements in the balanced accuracy amount to 2.51 percentage points (logistic regression), 3.03 percentage points (elastic net), 1.26 percentage points (random forest), and 2.33 percentage points (neural network). 

The increases in ROI amount to 46.31 percentage points (logistic regression), 48.24 percentage points (elastic net), 29.62 percentage points (random forest), and 52.25 percentage points (neural network). The increases in ROI highlight the economic value of incorporating textual self-descriptions when predicting startup success.

We also assess whether the improvement in prediction performance due to including textual self-descriptions is statistically significant. For this purpose, we utilize McNemar's test comparing the predictions with and without textual self-descriptions (while including fundamental variables). Here, we find that the performance increase is statistically significant at common significance thresholds for all considered machine learning classifiers, , logistic regression (χ^2-statistic = 21.70; p < 0.01), elastic net (χ^2-statistic = 25.55; p < 0.01), random forest (χ^2-statistic = 5.95; p < 0.05), and neural network (χ^2-statistic = 11.00; p < 0.01). In sum, the improvements from using textual self-descriptions are achieved consistently across all classifiers. The results thus confirm that textual self-descriptions have predictive power and thus are of operational value. 





For comparison, we also report the prediction performance of machine learning that is fed solely with textual self-descriptions. Here, the majority vote and random vote as naïve baselines are again outperformed by a considerable margin. Furthermore, we observe that the prediction performance of using only textual self-descriptions is comparable but slightly inferior to the prediction performance obtained by using only fundamental variables. For instance, for the neural network, the AUROC is 80.60 % for a model with only fundamental variables vs. an AUROC of 77.24 % for a model with only textual self-descriptions. 




 



§ CROSS-CORRELATION OF FUNDAMENTAL VARIABLES



The cross-correlation of the fundamental variables is shown in <Ref>. Strong correlations are observed, for example, between variables indicating the number of degrees of the founders (, founders degree count maximum and mean). While collinearity might affect correct estimates in explanatory analysis, it is beneficial for machine learning. The reason is that strong correlations often yield more powerful classifiers <cit.>. 


[ph!]
Cross-correlations of fundamental variables.

round-mode=places,round-precision=2

    Variable     1     2      3     4     5     6     7     8     9     10     11     12     13     14     15     16     17     18     19     20     21     22

1     Age     1                                                                                                          
 
    2     Has email     -0.02     1                                                                                                     
 
    3     Has phone      0.03      0.29     1                                                                                                
 
    4     Has Facebook      0.05      0.22      0.11     1                                                                                           
 
    5     Has Twitter      0.07      0.21      0.06      0.54     1                                                                                      
 
    6     Has LinkedIn     -0.10      0.19      0.12      0.23      0.28     1                                                                                 
 
    7     Number of investment rounds      0.24      0.01     -0.04      0.05      0.05      0.02     1                                                                            
 
    8     Raised funding      0.03      0.01      0.04     -0.01     -0.01      0.02      0.14     1                                                                       
 
    9     Last round raised funding      0.01      0.00      0.03     -0.03     -0.02      0.01      0.07      0.93     1                                                                  
 
   10     Last round post money evaluation      0.01      0.01      0.01      0.01      0.01      0.01      0.07      0.10      0.02     1                                                             
 
   11     Last round time lag      0.32     -0.08     -0.06     -0.02     -0.04     -0.11      0.68      0.06      0.05      0.02     1                                                        
 
   12     Investor count      0.16      0.06     -0.02      0.08      0.09      0.09      0.66      0.17      0.06      0.09      0.33     1                                                   
 
   13     Last round investor count      0.10      0.03     -0.04      0.04      0.06      0.05      0.48      0.09      0.10      0.02      0.38      0.62     1                                              
 
   14     Known investor count      0.22      0.00     -0.04      0.04      0.04      0.01      0.91      0.15      0.08      0.06      0.70      0.69      0.55     1                                         
 
   15     Last round known investor count      0.22     -0.01     -0.05      0.04      0.03      0.00      0.92      0.14      0.09      0.05      0.74      0.63      0.57      0.98     1                                    
 
   16     Founders count      0.02      0.11     -0.09      0.12      0.19      0.17      0.26      0.05      0.03      0.02      0.11      0.26      0.20      0.25      0.24     1                               
 
   17     Founders different country count      0.01      0.10     -0.09      0.12      0.18      0.16      0.23      0.03      0.02      0.01      0.09      0.22      0.17      0.21      0.21      0.91     1                          
 
   18     Founders male count      0.02      0.10     -0.08      0.11      0.18      0.17      0.26      0.06      0.04      0.02      0.11      0.26      0.20      0.25      0.24      0.97      0.88     1                     
 
   19     Founders female count      0.00      0.09     -0.08      0.12      0.17      0.13      0.18      0.02      0.01      0.01      0.06      0.18      0.15      0.17      0.17      0.79      0.82      0.64     1                
 
   20     Founders degree count total      0.01      0.10     -0.07      0.11      0.16      0.18      0.25      0.06      0.03      0.02      0.10      0.27      0.21      0.24      0.24      0.77      0.71      0.74      0.63     1           
 
   21     Founders degree count maximum      0.01      0.10     -0.07      0.11      0.17      0.18      0.24      0.05      0.02      0.02      0.09      0.25      0.19      0.23      0.22      0.78      0.79      0.75      0.70      0.94     1      
 
   22     Founders degree count mean      0.00      0.10     -0.07      0.11      0.17      0.18      0.23      0.04      0.02      0.02      0.09      0.24      0.19      0.22      0.22      0.78      0.80      0.75      0.71      0.91      0.99     1 
  

20lStated: Pearson correlation coefficient
    4rN = 20,172











