Some Work and Some Play: Microscopic and Macroscopic Approaches to Labor and Leisure
[['When suitably free , humans and other animals divide their limited time between work , i . e . , performing employer-defined tasks remunerated by rewards such as money or food , and leisure , i . e . , activities pursued for themselves that appear to confer intrinsic benefit .', 'The division of time provides insights into these quantities and their interaction , and has been addressed by both microeconomics and behavioral psychology .', 'Microeconomic labor supply theorists [1] have adopted a normative perspective , formulating what a rational agent should do .', 'Accounts from behavioral psychology have been descriptive , detailing how subjects allocate their time , for example , proportionally to the relative payoffs from work and leisure [2]–[8] .', 'Common to these approaches is the coarse , macroscopic timescale at which behavior is characterised , focusing on average times spent in work and leisure .', 'By contrast , microscopic analyses characterise the fine temporal topography of work and leisure choices , and so offer a foundation for examining , rather than averaging away , rich psychological and neural processes .', 'Tying microscopic and macroscopic choices together is known to be difficult in general [9] , because the former involves a much more elaborate state space than the latter .', 'Here , we build an approximately optimal stochastic control theoretic model of decision-making at a microscopic level .', 'We show how averaging over the microscopic choices yields a characterizable superset of traditional macroscopic theories , and casts the assumptions necessary for the latter to capture partial allocation in a different light .', 'We make the novel prediction that partial allocation requires neither stochastic choices ( as generally assumed by accounts from behavioral psychology ) nor the marginal utility of leisure to depend on the amount of work performed .', 'We use a simplification of a particularly stark labor task as a paradigm example to show how macroscopic and microscopic theories of the partial allocation of time between work and leisure can be tied .', 'We therefore do not attempt to model actual data from this task; a qualitative account is available in [10] .'], ['We consider a Cumulative Handling Time task [11] , [12] in which subjects must accumulate work up to a total time-period called the price ( see Table 1 for a list of symbols and their meanings ) to gain a reward .', 'The price and the objective strength of the reward are defined by the experimenter .', 'Note that the price is an experimenter determined time-period , hence we shall use “long” and “short” to denote its duration .', 'Subjects are free to distribute leisure bouts in between work bouts ( Fig . S1A ) .', 'The CHT controls both the ( average ) minimum inter-reward interval and the amount of work required to earn a reward .', 'This makes the CHT a generalisation of common schedules of reinforcement such as Fixed Ratio , or Variable Interval , which control one but not the other .', 'Reward and leisure are both assumed to enjoy a subjective worth .', 'We call these microscopic utilities to distinguish them from the macroscopic utilities used by traditional theories .', 'The microscopic utility of the former is called the ( subjective ) reward intensity ( , in arbitrary units ) ; the ratio of this to the price is called the payoff ( or in economic nomenclature , wage rate ) .', 'For simplicity , we consider the objective price , recognising that its subjective value may differ .', 'We explore different functional forms for the presumed microscopic utility of leisure .', 'This paradigm was originally developed in the context of rats pressing down an unweighted lever to gain non-satiating , brain stimulation reward ( BSR ) , or alternatively choosing leisure in the form of resting , grooming , exploring , etc .', 'However , as noted above , we do not model data , but rather consider an abstracted version of the task in order to concentrate on the relationship between microscopic and macroscopic descriptions .', 'The key macroscopic statistic is the Time Allocation ( ) : the proportion of trial time that the subject spends working [2] .', 'Fig . S1B shows example TAs for a typical subject .', 'As expected , the TA increases with reward intensity and decreases with price .', 'A microscopic analysis , as shown by ethograms in ( Fig . S1C ) , considers the detailed temporal topography of choice , recording when and for how long each act of work or leisure occurred .', 'Note that at intermediate payoffs , when partial allocation is most noticeable , subjects consume almost all leisure immediately after getting a reward , and then work continuously for each entire price [13] .', 'Labor supply theory and generalized matching average over the temporal topography shown in Fig . S1C ) .', 'By contrast , we follow [10] , [18] , [19] in formulating a so-called micro Semi-Markov Decision Process ( SMDP ) [20] , [21] ( Fig . 3A ) with actions , states , and utilities , for which policies ( i . e . , the stochastic choices of actions at states ) are quantified by the average reward per unit time accrued over the long run .', 'We formulated the general normative , microscopic theoretical framework in [10] .', 'Here we delineate a simplified model pertinent to the partial allocation problem .', 'By integrating the microscopic choices from our model , we can compare it with macroscopic descriptions such as the mountain model .', 'We saw that linear generates partial allocation with stochasticity .', 'It therefore generates smooth ( non-step function ) macroscopic time allocation curves as a function of both reward intensity and price .', 'Consequently , 3-dimensional relationships can be derived that are qualitatively similar to those specified by the mountain model ( when expressed in terms of subjective reward intensity , compare Fig . 6A with Fig . 5 ) .', 'However , when is non-linear , more complicated structures arise .', 'If the price is increased while holding the reward intensity fixed , the reward rate ( Eq .', '( 2 ) ) decreases hyperbolically and eventually asymptotes ( Fig . 7A ) .', 'Consequently , unlike the mean , the mode of the gamma-like distribution does not substantially increase with the price ( see Figs . 3C and 7B ) .', 'Since the mode determines the duration of the majority of leisure bouts , these do not increase substantially .', 'If the subject continues to work for the entire price duration ( Fig . 7C ) , then , surprisingly from the macroscopic perspective of the generalized matching model , the total work time and thus the TA will increase , rather than decrease with the price ( Figs . 6B and 7A , lower panel ) .', 'This prediction is readily amenable to experimental test .', 'Since for linear , leisure durations are governed by substantially changing means and not modes , TAs are in general smaller than for strictly concave , implying that higher payoffs are necessary to capture the entire TA range .'], ['We studied the problem of partial time allocation – when reward intensities and prices are not extreme , both animals and humans divide their time between work and leisure .', 'Traditional theories such as the microeconomic theory of labor supply , or accounts from behavioral psychology based on the generalised matching law , have characterised behavior at a macroscopic level , studying average times spent in work or leisure .', 'While labor supply approaches have studied choices within periods of time , these have been limited to maximising utility within these time windows [32]–and thus , still average times within these windows .', 'We proposed a normative , microscopic approach using the reinforcement learning framework of Semi-Markov Decision Processes .', 'Although we applied it to the labor-leisure tradeoff , this is actually a more general theoretical framework for temporally relevant decision-making .', 'By integrating the microscopic choices of our model over time , we were able to account for the nature of macroscopic partial allocation .', 'We showed how assumptions about microscopic and macroscopic quantities relate .', 'In labor supply theory , the marginal utility of leisure may ( although not necessarily ) depend on the amount of work ( or rewards ) consumed , and ( unlike in the behavioral data ) choices are classically deterministic .', 'We considered a stochastic policy of the same form as emerges for standard random utility models , but directed at microscopic , rather than macroscopic , choices .', 'Macroscopic random utility theory considers stochasticity to be due to unobservable noise , which is added to the representation of utility .', 'The subject chooses the combination of cumulative work and leisure times that maximizes this net utility ( including the noise term ) .', 'If the noise is assumed to be Gumbel distributed ( i . e . drawn from an extreme value distribution of type I ) , then the probability of choosing the optimal combination is a softmax .', 'The softmax function that we employ is over microscopic durations , and arises from an ( equivalently arbitrary ) assumption that subjects have a taste for entropic policies .', 'Randomness is thus directly built into the fabric of our model , rather than being an afterthought .', 'It generates partial allocation even when the marginal microscopic utility of leisure is independent of work .', 'Previous exercises attempting to link macroscopic static and dynamic frameworks have not been generally successful [9] .', 'Optimal choice in a dynamic context generally depends on the microscopic state , whose evolution is invisible at a macroscopic level .', 'This allows the macroscopic average choice obtained after integrating out such states ( i . e . , the average choice under the stationary distribution ) to appear counterintuitive , possibly even violating rationality constraints .', 'In our case , the key feature of the microscopic state is implicit in the non-memorylessness of the policies allowed in an SMDP – e . g . , that the hazard function governing the probability a leisure bout will end a certain time after it begun is not independent of time .', 'An example of the problems comes from observing that time allocation to working under conventional macroscopic labor supply accounts generally increases with reward and decreases with price .', 'Something similar is true of the macroscopic , mountain-like , consequence of generalized matching .', 'We showed in our framework that , although this can be true , it is nevertheless the case that for certain non-linearities , the time allocated to working can increase rather than decrease as the price increases , yielding complicated 3-dimensional relationships and non-monotonic contours that elude the mountain model .', 'We thus derived a transparent link between microscopic and macroscopic frameworks .', 'Whereas animals have been previously shown consistently to work more when work-requirements are greater ( one idea is that this arises from sunk costs [33] , [34] ) , the apparent anomaly discussed here only occurs at longer prices and is due to the form of the microscopic utility of leisure .', 'This is an obvious candidate for empirical investigation [35] .', 'Non-linear benefit of leisure functions can also lead to partial allocation for deterministic choices .', 'This applies even for functions that differ from those common in labor supply theory in virtue of satisfying independence between the microscopic utilities of working and engaging in leisure .', 'Of course , the marginal microscopic utility of leisure might depend on work or rewards – for instance due to fatigue or satiation .', 'However , carefully eliminating such dependencies ( by , e . g . , allowing subjects sufficient rest inbetween trials , and using non-satiating rewards like BSR ) may provide an avenue to quantify aspects of the microscopic utility of leisure empirically .', 'This should help reveal why and how subjects partially allocate their time .', 'It would then be natural to extend the study to considerations of effort , fatigue and cognitive computational costs [36]–[40] ( e . g . from holding down weighted levers or performing cognitively demanding tasks ) and the effects of manipulating motivational state [12] , [41] , [42] .', 'It is by taking advantage of the greater precision available from the detailed topography of work and leisure that we may hope to gain insight into these most important details .', 'Although previous work has described aspects of this topography [37] , [43] , our precise control theoretic formalization could offer enrichment .', 'The utilities considered in macroscopic labor supply theory are ordinal , whereas the microscopic utilities used in our framework are cardinal and , by analogy with quantities investigated in discrete choice paradigms [22]–[24] , open for direct neural investigation .', 'One of the key goals of our work is to provide a formal framework within which this can happen .', 'Finally , our work provides a foundation for studying critical psychological processes and neural computations at an appropriate timescale .', 'Real-time or quasi-real-time recording methods in routine use in neuroscience such as electrophysiology , large-scale imaging , or fast-scan cyclic voltammetry allow us to correlate the activity of neural populations or concentrations of neuromodulators with the execution of behaviors .', 'Likewise , fast causal manipulations via such methods as optogenetics allow the circuits governing these behaviors to be probed in a highly selective manner .', 'There is an evident mismatch between the microscopic timescale over which these methods operate and the macroscopic timescales over which', '( a ) behavior has often been characterised; and', '( b ) the quantities such as costs and benefits which underpin the pertinence of the behavior have been defined .', 'Our normative microscopic account may therefore provide an illuminating framework within which to build explanations that span multiple levels .'], ['See Micro-SMDP methods in Text S1 .']]
['Introduction', 'Results', 'Discussion', 'Methods']
['Given the option , humans and other animals elect to distribute their time between work and leisure , rather than choosing all of one and none of the other .', 'Traditional accounts of partial allocation have characterised behavior on a macroscopic timescale , reporting and studying the mean times spent in work or leisure .', 'However , averaging over the more microscopic processes that govern choices is known to pose tricky theoretical problems , and also eschews any possibility of direct contact with the neural computations involved .', 'We develop a microscopic framework , formalized as a semi-Markov decision process with possibly stochastic choices , in which subjects approximately maximise their expected returns by making momentary commitments to one or other activity .', 'We show macroscopic utilities that arise from microscopic ones , and demonstrate how facets such as imperfect substitutability can arise in a more straightforward microscopic manner .']
['Dividing limited time between work and leisure when both are attractive is a common everyday decision .', 'Rather than doing one exclusively , humans and other animals distribute their time between both .', 'Traditional explanations of this phenomenon have studied the macroscopic average times spent in both .', 'By contrast , we develop a microscopic framework in which we can model the real-time decisions that underpin these averages .', "In the framework , subjects' choices are approximately optimal , according to a natural , microscopic , utility function .", 'We show that the assumptions of previous theories are not necessary for partial allocation to be optimal , and show possibilities and limits to the integration of macroscopic and microscopic views .', 'Our approach opens new vistas onto the real-time processes underlying cost-benefit decision-making .']
['psychology', 'ecology and environmental sciences', 'social sciences', 'economic models', 'labor economics', 'ecology', 'behavior', 'ecological economics', 'biology and life sciences', 'microeconomics', 'computational biology', 'neuroscience', 'economics']