multi solving multiple machine learning tasks at the same time multitask learning mtl is a subfield of machine learning in which multiple learning tasks are solved at the same time while exploiting commonalities and differences across tasks this can result in improved learning efficiency and prediction accuracy for the taskspecific models when compared to training the models separately in a widely cited 1997 paper rich caruana gave the following characterizationmultitask learning is an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias it does this by learning tasks in parallel while using a shared representation what is learned for each task can help other tasks be learned better in the classification context mtl aims to improve the performance of multiple classification tasks by learning them jointly one example is a spamfilter which can be treated as distinct but related classification tasks across different users to make this more concrete consider that different people have different distributions of features which distinguish spam emails from legitimate ones for example an english speaker may find that all emails in russian are spam not so for russian speakers yet there is a definite commonality in this classification task across users for example one common feature might be text related to money transfer solving each users spam classification problem jointly via mtl can let the solutions inform each other and improve performance multitask learning works because regularization induced by requiring an algorithm to perform well on a related task can be superior to regularization that prevents overfitting by penalizing all complexity uniformly one situation where mtl may be particularly helpful is if the tasks share significant commonalities and are generally slightly under sampled methods task grouping and overlap within the mtl paradigm information can be shared across some or all of the tasks depending on the structure of task relatedness one may want to share information selectively across the tasks for example tasks may be grouped or exist in a hierarchy or be related according to some general metric suppose as developed more formally below that the parameter vector modeling each task is a linear combination of some underlying basis similarity in terms of this basis can indicate the relatedness of the tasks for example with sparsity overlap of nonzero coefficients across tasks indicates commonality a task grouping then corresponds to those tasks lying in a subspace generated by some subset of basis elements where tasks in different groups may be disjoint or overlap arbitrarily in terms of their bases exploiting unrelated tasks one can attempt learning a group of principal tasks using a group of auxiliary tasks unrelated to the principal ones in many applications joint learning of unrelated tasks which use the same input data can be beneficial the reason is that prior knowledge about task relatedness can lead to sparser and more informative representations for each task grouping essentially by screening out idiosyncrasies of the data distribution novel methods which builds on a prior multitask methodology by favoring a shared lowdimensional representation within each task grouping have been proposed the programmer can impose a penalty on tasks from different groups which encourages the two representations to be orthogonal experiments on synthetic and real data have indicated that incorporating unrelated tasks can result in significant improvements over standard multitask learning methods transfer of knowledge related to multitask learning is the concept of knowledge transfer whereas traditional multitask learning implies that a shared representation is developed concurrently across tasks transfer of knowledge implies a sequentially shared representation large scale machine learning projects such as the deep convolutional neural network googlenet group online adaptive learning traditionally multitask learning and transfer of knowledge are applied to stationary learning settings their extension to nonstationary environments is termed group online adaptive learning goal sharing information could be particularly useful if learners operate in continuously changing environments because a learner could benefit from previous experience of another learner to quickly adapt to their new environment such groupadaptive learning has numerous applications from predicting financial timeseries through content recommendation systems to visual understanding for adaptive autonomous agents mathematics reproducing hilbert space of vector valued functions rkhsvv the mtl problem can be cast within the context of rkhsvv a complete inner product space of vectorvalued functions equipped with a reproducing kernel in particular recent focus has been on cases where task structure can be identified via a separable kernel described below the presentation here derives from ciliberto et al 2015 rkhsvv concepts suppose the training data set is s t x i t y i t i 1 n t displaystyle mathcal stxityiti1nt with x i t x displaystyle xitin mathcal x y i t y displaystyle yitin mathcal y where t indexes task and t 1 t displaystyle tin 1t let n t 1 t n t displaystyle nsum t1tnt in this setting there is a consistent input and output space and the same loss function l r r r displaystyle mathcal lmathbb r times mathbb r rightarrow mathbb r for each task this results in the regularized machine learning problem min f h t 1 t 1 n t i 1 n t l y i t f t x i t f h 2 displaystyle min fin mathcal hsum t1tfrac 1ntsum i1ntmathcal lyitftxitlambda fmathcal h2 1 where h displaystyle mathcal h is a vector valued reproducing kernel hilbert space with functions f x y t displaystyle fmathcal xrightarrow mathcal yt having components f t x y displaystyle ftmathcal xrightarrow mathcal y the reproducing kernel for the space h displaystyle mathcal h of functions f x r t displaystyle fmathcal xrightarrow mathbb r t is a symmetric matrixvalued function x x r t t displaystyle gamma mathcal xtimes mathcal xrightarrow mathbb r ttimes t such that x c h displaystyle gamma cdot xcin mathcal h and the following reproducing property holds f x c r t f x c h displaystyle langle fxcrangle mathbb r tlangle fgamma xcdot crangle mathcal h 2 the reproducing kernel gives rise to a representer theorem showing that any solution to equation 1 has the form f x t 1 t i 1 n t x x i t c i t displaystyle fxsum t1tsum i1ntgamma xxitcit 3 separable kernels the form of the kernel induces both the representation of the feature space and structures the output across tasks a natural simplification is to choose a separable kernel which factors into separate kernels on the input space x and on the tasks 1 t displaystyle 1t in this case the kernel relating scalar components f t displaystyle ft and f s displaystyle fs is given by x i t x j s k x i x j k t s t k x i x j a s t textstyle gamma xitxjskxixjktstkxixjast for vector valued functions f h displaystyle fin mathcal h we can write x i x j k x i x j a displaystyle gamma xixjkxixja where k is a scalar reproducing kernel and a is a symmetric positive semidefinite t t displaystyle ttimes t matrix henceforth denote s t psd matrices r t t displaystyle sttextpsd matricessubset mathbb r ttimes t this factorization property separability implies the input feature space representation does not vary by task that is there is no interaction between the input kernel and the task kernel the structure on tasks is represented solely by a methods for nonseparable kernels is an current field of research for the separable case the representation theorem is reduced to f x i 1 n k x x i a c i textstyle fxsum i1nkxxiaci the model output on the training data is then kca where k is the n n displaystyle ntimes n empirical kernel matrix with entries k i j k x i x j textstyle kijkxixj and c is the n t displaystyle ntimes t matrix of rows c i displaystyle ci with the separable kernel equation 1 can be rewritten as min c r n t v y k c a t r k c a c displaystyle min cin mathbb r ntimes tvykcalambda trkcactop p where v is a weighted average of l applied entrywise to y and kca the weight is zero if y i t displaystyle yit is a missing observation note the second term in p can be derived as follows f h 2 i 1 n k x i a c i j 1 n k x j a c j h i j 1 n k x i a c i k x j a c j h bilinearity i j 1 n k x i x j a c i c j r t reproducing property i j 1 n k x i x j c i a c j t r k c a c displaystyle beginalignedfmathcal h2leftlangle sum i1nkcdot xiacisum j1nkcdot xjacjrightrangle mathcal hsum ij1nlangle kcdot xiacikcdot xjacjrangle mathcal htextbilinearitysum ij1nlangle kxixjacicjrangle mathbb r ttextreproducing propertysum ij1nkxixjcitop acjtrkcactop endaligned known task structure task structure representations there are three largely equivalent ways to represent task structure through a regularizer through an output metric and through an output mapping regularizerwith the separable kernel it can be shown below that f h 2 s t 1 t a t s f s f t h k textstyle fmathcal h2sum st1tatsdagger langle fsftrangle mathcal hk where a t s displaystyle atsdagger is the t s displaystyle ts element of the pseudoinverse of a displaystyle a and h k displaystyle mathcal hk is the rkhs based on the scalar kernel k displaystyle k and f t x i 1 n k x x i a t c i textstyle ftxsum i1nkxxiattop ci this formulation shows that a t s displaystyle atsdagger controls the weight of the penalty associated with f s f t h k textstyle langle fsftrangle mathcal hk note that f s f t h k textstyle langle fsftrangle mathcal hk arises from f t h k f t f t h k textstyle ftmathcal hklangle ftftrangle mathcal hk proof f h 2 i 1 n x i t i c i t i j 1 n x j t j c j t j h i j 1 n c i t i c j t j x i t i x j t j i j 1 n s t 1 t c i t c j s k x i x j a s t i j 1 n k x i x j c i a c j r t i j 1 n k x i x j c i a a a c j r t i j 1 n k x i x j a c i a a c j r t i j 1 n s t 1 t a c i t a c j s k x i x j a s t s t 1 t a s t i 1 n k x i a c i t j 1 n k x j a c j s h k s t 1 t a s t f t f s h k displaystyle beginalignedfmathcal h2leftlangle sum i1ngamma xiticdot citisum j1ngamma xjtjcdot cjtjrightrangle mathcal hsum ij1nciticjtjgamma xitixjtjsum ij1nsum st1tcitcjskxixjastsum ij1nkxixjlangle ciacjrangle mathbb r tsum ij1nkxixjlangle ciaadagger acjrangle mathbb r tsum ij1nkxixjlangle aciadagger acjrangle mathbb r tsum ij1nsum st1tacitacjskxixjastdagger sum st1tastdagger langle sum i1nkxicdot acitsum j1nkxjcdot acjsrangle mathcal hksum st1tastdagger langle ftfsrangle mathcal hkendaligned output metrican alternative output metric on y t displaystyle mathcal yt can be induced by the inner product y 1 y 2 y 1 y 2 r t displaystyle langle y1y2rangle theta langle y1theta y2rangle mathbb r t with the squared loss there is an equivalence between the separable kernels k i t displaystyle kcdot cdot it under the alternative metric and k displaystyle kcdot cdot theta under the canonical metric output mappingoutputs can be mapped as l y t y displaystyle lmathcal ytrightarrow mathcal tilde y to a higher dimensional space to encode complex structures such as trees graphs and strings for linear maps l with appropriate choice of separable kernel it can be shown that a l l displaystyle altop l task structure examples via the regularizer formulation one can represent a variety of task structures easily letting a i t 1 t 1 1 textstyle adagger gamma itgamma lambda frac 1tmathbf 1 mathbf 1 top where i t displaystyle it is the txt identity matrix and 1 1 textstyle mathbf 1 mathbf 1 top is the txt matrix of ones is equivalent to letting control the variance t f t f h k textstyle sum tftbar fmathcal hk of tasks from their mean 1 t t f t textstyle frac 1tsum tft for example blood levels of some biomarker may be taken on t patients at n t displaystyle nt time points during the course of a day and interest may lie in regularizing the variance of the predictions across patients letting a i t m displaystyle adagger alpha italpha lambda m where m t s 1 g r i t s g r displaystyle mtsfrac 1grmathbb i tsin gr is equivalent to letting displaystyle alpha control the variance measured with respect to a group mean r t g r f t 1 g r s g r f s displaystyle sum rsum tin grftfrac 1grsum sin grfs here g r displaystyle gr the cardinality of group r and i displaystyle mathbb i is the indicator function for example people in different political parties groups might be regularized together with respect to predicting the favorability rating of a politician note that this penalty reduces to the first when all tasks are in the same group letting a i t l displaystyle adagger delta itdelta lambda l where l d m displaystyle ldm is the laplacian for the graph with adjacency matrix m giving pairwise similarities of tasks this is equivalent to giving a larger penalty to the distance separating tasks t and s when they are more similar according to the weight m t s displaystyle mts ie displaystyle delta regularizes t s f t f s h k 2 m t s displaystyle sum tsftfsmathcal hk2mts all of the above choices of a also induce the additional regularization term t f h k 2 textstyle lambda sum tfmathcal hk2 which penalizes complexity in f more broadly learning tasks together with their structure learning problem p can be generalized to admit learning task matrix a as follows min c r n t a s t v y k c a t r k c a c f a displaystyle min cin mathbb r ntimes tain stvykcalambda trkcactop fa q choice of f s t r displaystyle fstrightarrow mathbb r must be designed to learn matrices a of a given type see special cases below optimization of q restricting to the case of convex losses and coercive penalties ciliberto et al have shown that although q is not convex jointly in c and a a related problem is jointly convex specifically on the convex set c c a r n t s t r a n g e c k c r a n g e a displaystyle mathcal ccain mathbb r ntimes ttimes strangectop kcsubseteq rangea the equivalent problem min c a c v y k c t r a c k c f a displaystyle min cain mathcal cvykclambda tradagger ctop kcfa r is convex with the same minimum value and if c r a r displaystyle crar is a minimizer for r then c r a r a r displaystyle crardagger ar is a minimizer for q r may be solved by a barrier method on a closed set by introducing the following perturbation min c r n t a s t v y k c t r a c k c 2 i t f a displaystyle min cin mathbb r ntimes tain stvykclambda tradagger ctop kcdelta 2itfa s the perturbation via the barrier 2 t r a displaystyle delta 2tradagger forces the objective functions to be equal to displaystyle infty on the boundary of r n t s t displaystyle rntimes ttimes st s can be solved with a block coordinate descent method alternating in c and a this results in a sequence of minimizers c m a m displaystyle cmam in s that converges to the solution in r as m 0 displaystyle delta mrightarrow 0 and hence gives the solution to q special cases spectral penalties dinnuzo et al suggested setting f as the frobenius norm t r a a displaystyle sqrt tratop a they optimized q directly using block coordinate descent not accounting for difficulties at the boundary of r n t s t displaystyle mathbb r ntimes ttimes st clustered tasks learning jacob et al suggested to learn a in the setting where t tasks are organized in r disjoint clusters in this case let e 0 1 t r displaystyle ein 01ttimes r be the matrix with e t r i task t group r displaystyle etrmathbb i texttask tin textgroup r setting m i e e t displaystyle miedagger et and u 1 t 11 displaystyle ufrac 1tmathbf 11 top the task matrix a displaystyle adagger can be parameterized as a function of m displaystyle m a m m u b m u i m displaystyle adagger mepsilon muepsilon bmuepsilon im with terms that penalize the average between clusters variance and within clusters variance respectively of the task predictions m is not convex but there is a convex relaxation s c m s t i m s t t r m r displaystyle mathcal scmin stimin stland trmr in this formulation f a i a m a m s c displaystyle famathbb i amin amin mathcal sc generalizations nonconvex penalties penalties can be constructed such that a is constrained to be a graph laplacian or that a has low rank factorization however these penalties are not convex and the analysis of the barrier method proposed by ciliberto et al does not go through in these cases nonseparable kernels separable kernels are limited in particular they do not account for structures in the interaction space between the input and output domains jointly future work is needed to develop models for these kernels applications spam filtering using the principles of mtl techniques for collaborative spam filtering that facilitates personalization have been proposed in large scale open membership email systems most users do not label enough messages for an individual local classifier to be effective while the data is too noisy to be used for a global filter across all users a hybrid globalindividual classifier can be effective at absorbing the influence of users who label emails very diligently from the general public this can be accomplished while still providing sufficient quality to users with few labeled instances web search using boosted decision trees one can enable implicit data sharing and regularization this learning method can be used on websearch ranking data sets one example is to use ranking data sets from several countries here multitask learning is particularly helpful as data sets from different countries vary largely in size because of the cost of editorial judgments it has been demonstrated that learning various tasks jointly can lead to significant improvements in performance with surprising reliability roboearth in order to facilitate transfer of knowledge it infrastructure is being developed one such project roboearth aims to set up an open source internet database that can be accessed and continually updated from around the world the goal is to facilitate a cloudbased interactive knowledge base accessible to technology companies and academic institutions which can enhance the sensing acting and learning capabilities of robots and other artificial intelligence agents software package the multitask learning via structural regularization malsar matlab package implements the following multitask learning algorithms meanregularized multitask learning multitask learning with joint feature selection robust multitask feature learning tracenorm regularized multitask learning alternating structural optimization incoherent lowrank and sparse learning robust lowrank multitask learning clustered multitask learning multitask learning with graph structures artificial intelligence artificial neural network automated machine learning automl evolutionary computation general game playing humanbased genetic algorithm kernel methods for vector output machine learning multitask optimization robot learning transfer learning the biosignals intelligence group at uiuc washington university at st louis depart of computer science software the multitask learning via structural regularization package online multitask learning toolkit omt a generalpurpose online multitask learning toolkit based on conditional random field models and stochastic gradient descent training c net 