How about orthogonality? How about differences between PLS algorithms?

When good pre­dic­tive abil­ity is the one and only goal, it is per­haps less impor­tant to think about orthog­o­nal­ity. The rea­son is sim­ple: we decide to only think about the pre­dicted val­ues and we don’t care about look­ing at any­thing else in the mod­els we have cre­ated. When talk­ing about dif­fer­ences between dif­fer­ent lin­ear algo­rithms like PLS (par­tial least squares), it is very tempt­ing to draw the erro­neous con­clu­sion that they are all the same just because the pre­dicted val­ues are sim­i­lar. In pre­dic­tive abil­ity, yes, but how about other model prop­er­ties? How about if one algo­rithm pre­dicts as well as oth­ers, but at the same time has bet­ter math­e­mat­i­cal properties?

If you only want good pre­dic­tions, don’t use lin­ear meth­ods like PLS. If you want some­thing non-orthog­o­nal, choose some­thing that is good at being non-orthog­o­nal. Use ANN (arti­fi­cial neural net­works) or other non-lin­ear black-box meth­ods or sup­port vec­tor machines or LOCAL, like the pro­fes­sion­als do. Select­ing a PLS-algo­rithm just because it has as a sim­i­lar pre­dic­tive abil­ity as other PLS algo­rithms does no make any sense. Like I said in the con­clu­sion of my my paper on a com­par­i­son of nine dif­fer­ent PLS algorithms:

There is no rea­son to let PLS stand for “Par­tial Lit­tle Squares” or “Par­tial Less Squares” when there is noth­ing to gain from it. Use only the numer­i­cally sta­ble algo­rithms and let PLS stand for “Par­tial Least Squares”.

This refers to the least squares of BOTH the X and the Y sides and the con­clu­sion still holds. The paper was very focused on reach­ing to a con­clu­sion to the rather dif­fi­cult ques­tion on which alorithms are the the best, and per­haps I focused too much on the numer­i­cal dif­fer­ences. I did empha­size that the dif­fer­ences relate to orthog­o­nal­ity prop­er­ties of the under­ly­ing latent vec­tors, but now I think that this point needs some more atten­tion. Why would orthog­o­nal­ity be impor­tant? Are sim­i­lar pre­dic­tions not enough? Actu­ally, I used to think so myself and I remem­ber that I thought: “Who cares?”.

I will now list some sim­ple rea­sons why orthog­o­nal­ity of math­e­mat­i­cal mod­els are important:

  1. If you look at the scores (the pro­jected data onto the model) and if you plot them in xy-scat­ter plots, you may pre­fer to have 90 degrees between the axes. If you use inac­cu­rate PLS algo­rithms, you can’t be sure it’s 90 degrees.
  2. If you like to cal­cu­late Maha­lanobis dis­tances between points, you may like to use euclid­ian dis­tances. Such dis­tances only hold if you have an orthog­o­nal coor­di­nate system.
  3. You may want to cal­cu­late the dis­tance to the model for new data. If the axes of the under­ly­ing model are twisted and not 90 degrees, you don’t know what you will get.
  4. You may like to use lin­ear alge­bra and rules of math­e­mat­ics to cre­ate new algo­rithms. Orthog­o­nal­ity is then impor­tant if you want to link yyour mod­els to old as well as new theory.

I am sure there are more rea­sons. Con­clu­sion. Use only the best PLS algo­rithms if you want to use PLS. Don’t look at only the pre­dic­tions to judge what is a bet­ter algo­rithm. Orthog­o­nal­ity is indeed impor­tant, but if you care only about pre­dic­tions, it is okay to for­get about orthog­o­nal­ity for a while. While doing so, you’d bet­ter switch to ANN or another non-lin­ear method that is good at being non-linear.

Leave a Comment

Your email address will not be published. Required fields are marked *

This website uses cookies. By continuing to use this site, you accept our use of cookies. 

Scroll to Top