Furthermore, the rescuing activity of DLK-1L was strongly attenuated by co-overexpression with DLK-1S ( Figures 1C and 1D, juEx2802, juEx2813). This inhibitory effect of DLK-1S was eliminated when the LZ domain was deleted from DLK-1S ( Figure S2C). However, expression of a kinase-dead mutant DLK-1S(K162A), in which the Lys162 at the ATP binding

site of the kinase domain was mutated to Ala ( Nakata et al., 2005), inhibited DLK-1L to a similar degree as did wild-type DLK-1S ( Figure S2C). These data suggest that the ability of DLK-1S to inhibit DLK-1L requires its LZ domain but not its kinase activity. As a further test for the role of DLK-1S, we expressed various DLK-1 constructs in the wild-type background ( Figure S2D). Overexpression Obeticholic Acid research buy of DLK-1L alone caused abnormal neuronal development, whereas overexpression of DLK-1(mini) gene had a much weaker effect. Removing intron 7 from DLK-1(mini), which would prevent production of DLK-1S, resulted in gain-of-function effects similar to DLK-1(L). Finally, to address whether transgenically expressed DLK-1S could interfere

with endogenous DLK-1L, we overexpressed DLK-1S in rpm-1(lf) single mutants and observed this website significant suppression of rpm-1(lf) phenotypes ( Figure S3A). Together, these analyses demonstrate that despite sharing identical kinase and LZ domains, DLK-1S is a potent inhibitor of DLK-1L function. If DLK-1S acts as an endogenous inhibitory isoform, how does DLK-1L become activated at all? Since DLK-1L and DLK-1S differ only in their C termini, we hypothesized that the C terminus of DLK-1L may contain elements important

for its kinase activation and that DLK-1S may act by preventing the interactions between the such elements and the kinase domain. To test this idea, we generated a series of DLK-1L variants in which the C terminus was either truncated or contained internal deletions (Supplemental Experimental Procedures) and assayed rescuing activity of these constructs in the dlk-1(lf); rpm-1(lf) double mutant strain ( Figure 2, Table S2). We found that a region of 25 amino acids from residues 856 to 881 in the DLK-1L C terminus was necessary for DLK-1L activity ( Figure 2, juEx3586). Remarkably, a construct lacking all of the DLK-1L C terminus except for aa 856–881 recapitulated the activity of the full-length DLK-1L ( Figure 2, juEx3657), suggesting that this region is sufficient for DLK-1L regulation. Upon closer inspection of the amino acid sequences, we found a six residue motif SDGLSD (aa 874–879, hereafter referred to as the hexapeptide) that is completely conserved between C. elegans DLK-1 and vertebrate MAP3K13/LZK ( Figure 3A); the remainder of the C termini of these kinases show little sequence conservation. Moreover, dlk-1(ju620), a strong loss-of-function mutation, results in a missense alteration (G870E) adjacent to this hexapeptide.

7 T (see Logothetis et al., 1999). In each session, SE-EPI and GE anatomical reference images were acquired with the same slice orientation as the functional images. For the GE anatomical reference, which was used for quick visualization during experiments, FOV was 12.8 × 9.6 cm2, matrix size 256 × 256, slice thickness 1 mm, TE 10 ms, and TR 750 ms. For the SE-EPI anatomical reference, which was used as an intermediate to accurately coregister the statistical map with the MDEFT image, a 16 segment SE-EPI was VE-821 clinical trial acquired (see Goense et al., 2008).

The matrix was 256 × 192, bandwidth 60–159 kHz, spatial resolution 0.5 × 0.5 mm2, slice thickness 1 mm, TE 62 ms, and TR 4 s. For field mapping, two 3D FLASH images were acquired with FOV 12.8 × 9.6 × 9.6 cm3 and matrix 128 × 128 × 64, resulting in a resolution of 1 × 0.75 × 1.5 mm3. TEs were 4.9 and 5.9 ms, TR was 50 ms, and flip angle was 15°. Data were field-map corrected as described previously (Goense et al., 2008). For anesthetized experiments a 12 cm custom-made quadrature RF coil was used that covered the entire brain. Images were acquired check details by using a four segment SE-EPI. The FOV was 8.0 × 7.2 cm2, with a matrix size of 80 × 72, yielding a final resolution of 1.0 × 1.0 mm2. The slices were acquired along the temporal lobe, and 22–25 slices with a thickness of 2 mm were typically needed to cover the entire brain.

TE was 40 ms and TR was 2 s per segment, yielding a final temporal resolution of 8 s per volume. Data were acquired in a single session (experiment day) for each animal, which amounted to 1800 volumes for N08, 2160 volumes for C06, and 1368 volumes for L04. For anatomical reference a 16 segment SE-EPI was acquired in each scanning session. The matrix

was 192 × 176 and the FOV was 8.0 × 7.2 cm2 with 1 mm slice thickness. TE was 62 ms and TR was 3 s. Reference anatomical scans and the 3D FLASH for field mapping STK38 were acquired by using the same parameters as in awake experiments. EPI images were reconstructed by using Bruker ParaVision 4.0 software. Data were analyzed by using custom-written software in MATLAB (The MathWorks, Natick, MA, USA), SPM 2 and SPM 5 (Wellcome Department of Cognitive Neurology, London, UK [Friston et al., 1995]), and Caret 5.9 (Washington University, St. Louis, USA [Van Essen et al., 2001]). Data from awake monkeys were processed following the methods described in Goense et al., 2008. Images were realigned, field-map corrected, and coregistered with the anatomical image by using SPM 2. For anesthetized animals similar procedures were used. Images were smoothed by using a 3 mm (awake) or 2 mm (anesthetized) full width at half maximum Gaussian kernel. Statistical analysis was done in SPM 2 by using general linear model analysis with the default hemodynamic response function. Activation was thresholded (at a significance level of p < 0.

, 1998). Thus, VEGF chemoattracts commissural axons www.selleckchem.com/products/iwr-1-endo.html through Flk1, with a negligible or redundant role for other Flk1 ligands (VEGF-C) or activators (Sema3E), or VEGF receptors (Npn1). Curiously, motor columns express Vegf mRNA, yet neither vessels nor commissural axons invade these structures ( James et al., 2009). It is possible that motor neurons make the Vegf message (mRNA), but do not secrete the protein from their cell body, and target it to their axonal compartment (as is thought to occur for Slit2; M.T.-L. and A. Jaworski, unpublished data). Another alternative

explanation is that additional signals prevent blood vessels and commissural axons from entering the motor columns. VEGF was originally discovered as a key angiogenic factor. Only subsequent studies revealed that this factor can affect neurons directly, independently of its angiogenic activity (Rosenstein et al., 2010, Ruiz de Almodovar et al., 2009, Ruiz de Almodovar et al., 2010 and Tam and Watts, 2010). In the developing spinal cord, VEGF orchestrates the formation of the neurovascular plexus and subsequent vessel sprouting from this plexus into the avascular AT13387 neural tube (James et al., 2009). Interestingly, however, even though vascularization of the neural tube occurs at the same time as commissural axon midline crossing, our conditional Flk1 inactivation

studies in commissural neurons and in vitro turning assays establish that VEGF chemoattracts

these axons independently of any VEGF-related vascular activity. To the best of our knowledge, this is the first report documenting an angiogenesis-independent effect of VEGF on axon guidance. The VEGFLacZ mouse line was kindly provided by A. Nagy and was previously described about ( Miquerol et al., 1999). VEGF-C knockout mice were previously described ( Karkkainen et al., 2004), and the VEGFlox/lox mouse line was kindly provided by D. Anderson and previously described ( Gerber et al., 1999). The transgenic Wnt1-Cre mouse line was kindly provided by A. McMahon. The transgenic Hoxa1-Cre mouse line was generated by A. Chedotal using a previously described cDNA ( Li and Lufkin, 2000). The Flk1lox/LacZ mouse line was generated by crossing Flk1lox/lox mice ( Haigh et al., 2003) with Flk1LacZ/+ mice ( Maes et al., 2010). For each transgenic line, WT littermate embryos were used. Wistar or Sprague Dawley rat embryos (E13) were used for explant outgrowth assays, purification of commissural neurons and for immuno-histochemistry. All animals were treated according to the guidelines approved by the Animal Care Committees of the K.U.Leuven (Belgium) and of the IRCM (Canada). Commissural neurons were prepared from the dorsal fifth of E13 rat neural tubes as described (Langlois et al., 2010 and Yam et al., 2009).

We found that there was a strong correlation between the strength of E-LTP expression (measured as E1 volume just prior to L2 stimulation) and the strength of the synaptic tag (measured as the volume of E1 3 hr after L2 stimulation thought to be proportional to the synaptic tag strength; Figure 3D). This indicates that the strength of the synaptic tag is tightly correlated with the level of E-LTP expression when GLU stimulation precedes

GLU+FSK stimulation. Under these conditions, the magnitudes of L-LTP at the spines given GLU stimulation were generally lower than that at spines given GLU+FSK stimulation (Figure 2K). On the other hand, when GLU stimulation followed GLU+FSK stimulation, the magnitude of L-LTP at the spine given

GLU stimulation (E2) tended to be similar to that of the spine given GLU+FSK stimulation (Figure 2F). Therefore, we hypothesized that PrPs NLG919 strengthen tags at spines stimulated later, but not earlier. To test this idea, we employed a subthreshold stimulus (each uncaging laser pulse lasts 1 ms instead of 4 ms) that, consistent with previously reported data (Harvey and Svoboda, 2007), did not cause a change in spine MLN0128 solubility dmso volume (Figure 3E). When this stimulus was given to a spine (S2) after GLU+FSK stimulation was given to another spine (L1), a significant increase in spine volume was seen at both L1 and S2 (Figure 3F). Similar to the L1-L2 stimulation (Figures 2C and 2D) and L1-E2 stimulation (Figure 2F) experiments, this growth depended on protein synthesis during L1 stimulation, but not S2 stimulation (data not shown). However, when the same subthreshold stimulus (S1) was given to one spine before GLU+FSK stimulation was given to a second spine (L2), no growth at S1 was seen, whereas L2 grew normally (Figure 3G). These data support the hypothesis that PrPs strengthen tags at spines stimulated after their production, but not those stimulated

earlier. Thus, though STC is temporally bidirectional, the two directions are different, not only because the lifetimes of the synaptic tag and the rate-limiting PrP are different, but also because L-LTP induction or expression seems to facilitate tag formation at PD184352 (CI-1040) spines stimulated later, but not at spines stimulated earlier. Having established STC at the single-spine level and revealed its asymmetric bidirectionality, we next examined the spatial span over which STC occurs, an issue central to the CPH (Govindarajan et al., 2006). The spatial relationship between spines participating in STC cannot be studied using field stimulation, since it results in the activation of many unidentified spines. But, it can be studied in our system by varying the distance between two spines both of which are given GLU+FSK stimulations (L1, L2) with only L2 stimulation conducted in the presence of anisomycin.

To examine cognitive functions in more detail, we extended our cognitive evaluation Bioactive Compound Library in vivo by assessing memory extinction. Memory extinction is a form of inhibitory learning that provides a basis for an adaptive control

of cognition and represents one of the key aspects of mental flexibility (Radulovic and Tronson, 2010, Herry et al., 2010 and Floresco and Jentsch, 2011). While it has been demonstrated that various stages of memory extinction require the recruitment of widespread brain domains, the molecular mechanisms regulating this process remain unclear (Lattal and Abel, 2001, Lattal et al., 2003, Fischer et al., 2004, Fischer et al., 2007, Sananbenesi et al., 2007, Radulovic and Tronson, 2010, Agis-Balboa et al., 2011, Bahari-Javan et al., 2012 and Tronson et al., 2012). In order to test the effects of Tet1 ablation on memory extinction, we performed extinction training using two groups of male littermate Tet1+/+ and Tet1KO mice following contextual fear conditioning as described earlier. Twenty-four hours

after fear conditioning, both Tet1+/+ and Tet1KO groups exhibited similar freezing levels (65%–75%) (p > 0.05; Figure 2D). After contextual fear memory test, Tet1+/+ and Tet1KO animals were placed into the same conditioning cages for a “massed” extinction trial (Cain et al., 2003 and Polack et al., 2012). Twenty-four Adriamycin hours later, memory extinction was assessed by scoring freezing events. Interestingly, we found that while control Tet1+/+ mice exhibited robust memory extinction (∼20% freezing after extinction training), Tet1KO mutants failed to display any memory extinction and retained an average freezing

level of about 60% (p < 0.01 control versus Tet1KO; Figure 2D). To extend cognitive evaluation of the Tet1KO mice, we assessed their hippocampus-dependent spatial reference memory using the Morris water Oxalosuccinic acid maze test (MWM) (Vorhees and Williams, 2006). Two groups of male littermate Tet1+/+ and Tet1KO mice were subjected to two training trials per day for 6 days, and the escape latency was scored for each trial. Probe trial was conducted 24 hr after the last day of training. We observed no significant differences between the groups during either the training or the probe trials (p > 0.05; p > 0.05; Figures 2E and 2F). To test spatial memory extinction, we then exposed control and Tet1KO mice used for spatial learning tests to extinction training in the MWM with the same spatial cues but with the platform removed (Zhang et al., 2011). We discovered that while Tet1+/+ mice demonstrated considerable memory extinction as evidenced by their progressively decreasing target quadrant occupancy (from about 35% on the probe trial to about 15% on the last day of extinction training), the Tet1KO animals persevered searching for the platform in the former target quadrant and did not show any decrease in quadrant preference (p > 0.05 for Tet1KO; p < 0.

, 2004). It should be noted that the specificity of the effect is controlled in Figures S6C and

S6D, demonstrating that the defect in synaptic endocytosis is critically dependent on the absence of LRRK and on the ability of EndoA to be phosphorylated. To assess whether the defects in FM1-43 dye uptake in the EndoA Talazoparib S75 phosphomutants are the result of reduced synaptic vesicle endocytosis, we recorded EJPs during a 10 min of 10 Hz stimulation paradigm; we analyzed boutonic ultrastructure and we recorded mEJPs, assays that when performed on Lrrk mutants show defects ( Figure 1). Under conditions of 2 mM external calcium, endoA+/+; endoAΔ4 controls, as well as endoA[S75A]/+; endoAΔ4 and endoA[S75D]/+; endoAΔ4, show similar EJP amplitudes during low-frequency stimulation, indicating normal synaptic transmission ( Figure S6E). In addition, FM1-43 internalized during a 5 min, 90 mM KCl stimulation paradigm is efficiently unloaded during a second stimulation period in both endoA+/+; endoAΔ4 controls and in animals expressing the EndoA phosphomutants,

indicating normal vesicle fusion under these conditions ( Figure S6F). However, both endoA[S75A]/+; endoAΔ4 and endoA[S75D]/+; endoAΔ4 fail to maintain neurotransmitter release during 10 min of 10 Hz stimulation, while endoA+/+; endoAΔ4 controls maintain release well ( Figures 7E and 7F). This defect is consistent with reduced synaptic vesicle recycling in the EndoA phosphomutants. Next, we performed TEM on stimulated third-instar larval boutons. Very similar to endoA hypomorphic mutants ( Guichet et al., 2002), synaptic vesicle number in endoA[S75A]/+; endoAΔ4, as well as in endoA[S75D]/+; selleck screening library endoAΔ4, is reduced and the number of cisternae is significantly increased compared to endoA+/+; endoAΔ4 controls ( Figures 7G–7J). Our data also suggest that the cisternae seen in endoA[S75A]/+; endoAΔ4 and endoA[S75D]/+; endoAΔ4 fuse with the membrane to release transmitters, as we observe larger mEJP amplitudes in endoA[S75A]/+; endoAΔ4 and in endoA[S75D]/+; endoAΔ4 but not in endoA+/+; endoAΔ4 controls ( Figure 7K and Figure S6G).

These defects are qualitatively similar to those observed in Lrrk mutants ( Figure 1) and collectively they suggest that animals expressing Thiamine-diphosphate kinase the EndoA S75 phosphomutants harbor synaptic vesicle recycling deficits that parallel the endocytic defect seen in endoA hypomorphic mutants ( Guichet et al., 2002). If endoA[S75D]/+; endoAΔ4 animals display reduced synaptic endocytosis, we expect expression of LRRK2G2019S that results in increased EndoA S75 phosphorylation in vivo to also lead to defects in synaptic vesicle endocytosis. We therefore performed FM1-43 dye uptake experiments in Drosophila expressing the kinase-active clinical mutant LRRK2G2019S. Similar to endoA mutants that express EndoA[S75D], we find that expression of LRRK2G2019S results in significantly reduced synaptic vesicle endocytosis, while expression of the kinase-dead LRRK2KD does not affect FM1-43 dye uptake ( Figure 8A).

8 NA, Olympus, Tokyo, Japan). Scanning and image acquisition were controlled by Fluoview software (Olympus); the average power delivered

to the brain was <50 mW. Imaging was carried out 72 hr before and after a retinal lesion (Figure 2A) at high resolution (1024 × 1024 pixels, 0.08 μm per pixel, 0.5 μm z step size). Imaging regions were repeatedly found by aligning the blood vessel pattern on the surface of the brain. Lower-resolution image stacks (512 × 512 pixels, 2.5 μm z step size) were acquired to visualize the dendritic and axonal branching pattern and the position of the soma, which, for the cells analyzed here, was confirmed to be located in cortical layer 1 or 2/3. Cells in the LPZ were chosen such that they were located at least 50 μm from the borders to avoid CP-673451 order any ambiguity. Cells located outside of the LPZ had cell bodies that were located >50 μm from the edge of the LPZ, as determined using intrinsic imaging within selleck products 3 days after the lesion. Distance from the border of the LPZ was calculated based on the position of the axon or, in the case of dendritic measurements, the cell body. High-resolution images were used for the analysis of dendritic spines and axonal boutons.

Image analysis was carried out using ImageJ (US National Institutes of Health, Bethesda, MD) and performed in three dimensional z stacks. Analysis of spines and boutons were restricted to cortical layers 1 and 2/3 (0–200 μm below the cortical surface). All protrusions, spines and filopodia were counted, including those that extended in the z axis. Spines and boutons were counted without the knowledge of experimental condition. Survival fraction is calculated at each time point as the number of boutons or spines still present that Casein kinase 1 were present at the first time point as a fraction of the

total number of initial boutons or spines. In total, we analyzed 16,259 boutons and 9633 spines over 9–12 time points. GAD65-GFP animals were deeply anaesthetized and cardially perfused first with saline (0.9% NaCl solution with 2.8 mg/liter heparin and 5 mg/liter lidocaine) and then with chilled 4% paraformaldehyde (4°C) for 30 min. Perfused brains were transferred to 30% sucrose solution for 2 days, after which they were sectioned coronally at 30 μm thickness. For analysis of excitatory and inhibitory synapses and inhibitory neuron cell type, the GFP signal in GABAergic neurons was amplified by immunofluorescence staining (chicken antibody to GFP, 1:1,000, Chemicon). GABAergic synapses were visualized by fluorescent labeling of VGAT (rabbit antibody to VGAT, 1:200, Synaptic Systems, Göttingen, Germany) and gephyrin (mouse antibody to gephyrin, 1:400, Synaptic Systems), as described previously (Wierenga et al., 2008). Glutmatergic synapses were visualized by fluorescent labeling of VGlut (rabbit α-VGlut1, 1:400, Synaptic Systems), as described previously (Becker et al., 2008).

In the hypoglossal nucleus, DSI of glycinergic inhibition to principal cells has been reported, suggesting that it

is not ZD1839 ic50 confined to GABAergic synapses (Mukhtarov et al., 2005). Although DSI generally lasts less than 5 min, eCBs have also been implicated in LTD of GABAergic inhibitory transmission (“iLTD”). In the lateral amygdala, low-frequency stimulation at 1 Hz, designed to release glutamate at synapses on the target neuron, was followed by a persistent depression of inhibitory transmission, which was sensitive to blocking either mGluR1 or CB1 receptors (Marsicano et al., 2002). The effect was potentiated by blocking anandamide degradation, implying that this eCB, rather than 2-AG, is involved (Azad et al., 2004). In contrast, iLTD in hippocampal pyramidal neurons is sensitive to blocking diacylglycerol lipase (Chevaleyre and Castillo, 2003), implicating 2-AG. Roles for presynaptic adenylate cyclase, inhibited by the αi/o limb of the CB1 signaling cascade, and for the active zone protein RIM1α, discriminate iLTD from DSI (Chevaleyre et al., 2007). This brief summary of CB1 receptor-mediated plasticity of inhibition focuses exclusively on activity-dependent eCB signaling. Signaling by eCBs may also be tonically active. For example, a CB1 antagonist was shown to increase GABA release from a subset of hippocampal CCK-positive interneurons (Losonczy et al., 2004), and similar results have been

reported in the hypothalamus (Oliet et al., 2007). These reports raise the possibility that CB1 receptor-mediated control of GABA release can be Small molecule library mw modulated up or down. However, most of the available CB1 antagonists act as inverse agonists (Kirilly et al., 2012). The observation that these compounds can increase GABA release could therefore be explained as relief from constitutive G protein-coupled receptor activity and therefore falls short of demonstrating basal occupancy of CB1 receptors by continued synthesis of eCBs. Several other retrograde factors have been reported to modulate GABA release

and lead to long-term changes because in inhibitory transmission. In the ventral tegmental area, nitric oxide can be synthesized in response to high-frequency stimulation of glutamatergic afferents innervating dopaminergic cells. Nitric oxide in this system appears to trigger LTP of GABAergic transmission (Nugent et al., 2007). This phenomenon coexists with eCB-mediated iLTD in the same dopaminergic neurons (Pan et al., 2008), and these long-term changes in GABAergic signaling are modulated by drugs of abuse and D2 dopamine receptors (Nugent et al., 2007; Pan et al., 2008). In the neonatal hippocampus, high-frequency stimulation of afferent fibers can lead to a presynaptic form of LTD of GABAergic transmission (McLean et al., 1996). The induction of this phenomenon has been attributed to GABAA receptor-mediated depolarization, leading to NMDA receptor-mediated Ca2+ influx.

The observation that correlations with the audio envelope decrease from early to higher order auditory processing areas is consistent with hierarchical models of auditory processing in which early auditory areas encode the lower level acoustic properties while higher order areas extract more abstract information (Chevillet et al., 2011; Hickok and Poeppel, 2004; Pallier et al., 2011). Previous work suggests that the capacity to accumulate information over time increases gradually from early sensory areas to higher

order perceptual and cognitive areas (Hasson et al., 2008; Lerner et al., 2011). Therefore, the gradient of weakening audio correlations within the STG should correspond to a gradient NVP-BKM120 clinical trial of lengthening temporal receptive windows (TRWs). To examine this relationship in our data, we defined the “TRW index” of each electrode as the difference of its repeat reliability for the intact and fine-scrambled movie clips. Thus, TRW(i) = rINTACT(i) − rFINE(i) where rINTACT(i) and rFINE(i) are the repeat reliability of the i-th electrode in the intact and fine-scrambled conditions ( Figure 4A, bottom inset). Within the STG, areas with longer TRWs exhibited RG7204 datasheet smaller audio correlations (Figures 4A–4C). A strong and significant anticorrelation was found between the TRW index of each electrode in the STG and the strength

of its coupling to the intact movie soundtrack (Figure 4B, black dashed line; r = −0.62,

p = 0.010, n = 16) and scrambled movie soundtrack (Figure 4B, green dashed line; r = −0.51, p = 0.04, n = 16). These results support the existence of a hierarchy of progressively longer TRWs within the STG. Areas nearer primary auditory cortex have shorter TRWs and are more sensitive to instantaneous transients of the stimulus, while areas with longer TRWs respond less to instantaneous stimulus transients, and more to the long-range temporal structure that is needed to follow the meaning of the story. Within the cerebral cortex as a whole, TRW values tended to be smaller in the vicinity of early sensory cortices and larger in higher order brain regions. Thus, by and large, the broadband response reliability in early auditory and Calpain visual regions was high at all scrambling levels (Figure 4C, blue). By contrast, in higher order areas nearer the anterior fusiform gyrus, the angular gyrus and frontal cortex (Figure 4C, red), the response reliability to the intact clip was larger than the reliability to the scrambled clips. Three visual electrodes exhibited significantly greater reliability for the scrambled movie than for the intact movie clip, possibly because the discontinuous fine-scrambled condition provided more opportunities to respond to the onset of a preferred stimulus. We confirmed the presence of a TRW gradient by clustering electrodes into regions of interest (ROIs) based on their anatomical location (Figure 5A).

Daw et al.’s finding that different subjects employ each system to a greater or lesser degree (Daw et al., 2011) might be seen as being evidence for the latter idea. Various suggestions have been made for how arbitration should proceed, but this is an area where much more work is necessary. One idea is that it should depend on the relative uncertainties of the systems, trading the noise induced by the calculational difficulties of model-based control off against the noise

induced by the sloth of learning of model-free control (Daw et al., Selleckchem PFI-2 2005). This provides a natural account of the emergence of habitual behavior (Dickinson, 1985), as in the latter noise decreases as knowledge accumulates. By this account, it could be the continual uncertainty induced by the changing mazes in Simon and Daw (2011) that led to the persistent dominance of model-based control.

Equally, the uncertainty associated with unforeseen circumstances might lead to the renewed dominance of model-based control, even after model-free control had asserted itself (Isoda and Hikosaka, 2011). A different idea suggested by Keramati et al. (2011) starts from the observation that model-free values are fast to compute but potentially inaccurate, whereas model-based ones are slow to compute but typically more accurate (Keramati et al., 2011). They consider a regime in which the model-based Selleck ERK inhibitor values are essentially perfect and then perform a cost/benefit analysis to assess whether the value of this perfect information is sufficient to make it worth acquiring expensively. The model-free controller’s uncertainty about

the relative values of the action becomes a measure of the potential benefit; and the opportunity cost of the time of calculation (quantified by the prevailing average reward rate (Niv et al., 2007) is a measure of the cost. A related suggestion involves integration of model-free and model-based values rather than selection and a different method of model-based calculation (Pezzulo et al., 2013). There is no unique form of model-free or model-based control and evidence hints that there are intermediate points on the spectrum between them. For instance, there are important differences between model-free control based on the predicted Casein kinase 1 long-run values of actions (as in Q-learning) (Watkins, 1989), or SARSA (Rummery and Niranjan, 1994), and actor-critic control (Barto et al., 1983). In the latter, for which there is some interesting evidence (Li and Daw, 2011), action choice is based on propensities that convey strictly less information than the long-run values of those actions. There are even ideas that the spiraling connections between the striatum and the dopamine system (Joel and Weiner, 2000 and Haber et al., 2000) could allow different forms of controller to be represented in different regions (Haruno and Kawato, 2006).