#! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/sh (not csh) to create: # Makefile # Readme # f2c.h # fig1.eps # fig2.eps # is.polymars # library.help # plot.polymars # polymars # polymars.tex # polymarsall.S # polymarsall.c # predict.polymars # print.polymars # summary.polymars # This archive created: Wed Jul 9 13:14:44 1997 export PATH; PATH=/bin:/usr/bin:$PATH if test -f 'Makefile' then echo shar: "will not over-write existing file 'Makefile'" else cat << \SHAR_EOF > 'Makefile' OBJECTS = polymarsall.o CFLAGS = polymars.o:polymarsall.c ld -d -r -lm -o polymars.o $(OBJECTS) polymars.so:polymarsall.c Splus SHLIB -o polymars.so polymarsall.c SHAR_EOF fi if test -f 'Readme' then echo shar: "will not over-write existing file 'Readme'" else cat << \SHAR_EOF > 'Readme' PolyMARS Version 1.0 ****************************************************************************** DOCUMENTATION More documentation about the polyMARS program is available on request. ****************************************************************************** FILES The unwrapped package contains the files Makefile is.polymars ReadMe polymars f2c.h plot.polymars library.help predict.polymars polymarsall.S print.polymars polymarsall.c summary.polymars polymars.tex fig1.eps fig2.eps ******************************************************************************** MAKEFILE The Makefile can make 2 files: polymars.so - needed for dyn.load.shared() polymars.o - needed for dyn.load() Depending on your compiler you may need to change some of the options in the Makefile. The compile command is either make polymars.so or make polymars.o ******************************************************************************** S-CODE The file polymars.S contains the code for the S functions that are related to the polymars program. They can be loaded by sourcing polymars.S from S. ******************************************************************************* HELPFILES The files polymars, predict.polymars, plot.polymars, is.polymars, summary.polymars and polymars.persp contain the helpfiles for the S functions. They should be placed in a directory .Data/.Help. This assumes that your system reads the helpfiles using some form of nroff. ******************************************************************************** LOADING THE OBJECT FILES You may need to edit the function polymars in polymarsall.S appropriately for loading. This function contains the line dyn.load.shared("/usr/local/splus33/library/polymars/polymars.so") It may need to be changed to dyn.load.shared("/where/your/files/are/polymars.so") or dyn.load("/where/your/files/are/polymars.o") ******************************************************************************** LIBRARY One possible way to install the programs is to put them in a library. How to do this: - find out where the S(Splus) code is located - if you don't know this, the easiest way is to type getenv("SHOME") inside S(Splus). Let's say that this is /usr/lang/splus - cd to this directory, and go further down to the library directory under this directory (cd /usr/lang/splus/library). - make a directory polymars under this directory (mkdir polymars) - move ALL the polymars related files into this directory - in /usr/lang/splus/library/polymars execute % mv library.help README % make polymars.so or make polymars.o % vi/emacs polymars.S as indicated above % mkdir .Data % ls .Data (this is in fact /usr/lang/splus/library/polymars/.Data) (Should contain polymarsall.S, polymarsall.c .....and no files that are not polymars related) % mkdir .Data/.Help % mv polymars predict.polymars plot.polymars is.polymars .Data/.Help % mv summary.polymars polymars.persp .Data/.Help (These are the helpfiles in /usr/lang/splus/library/polymars not the executables in /usr/lang/splus/library/polymars/.Data) This assumes that you have nroff. % S (or Splus) > source("polymarsall.S") > help.findsum.d(".Data") ( this sets up helpfiles for window-based help) > q() A user wanting to use the polymars program would now, once per session, execute the command library(polymars) which she/he could put in her/his .First function. ******************************************************************************** The C code contains a matrix inversion routine from lapack, all else is the work of the developers. Please note the copyright statement below. ******************************************************************************* * (c) Charles Kooperberg and Martin O'Connor 1997 * * This function is part of an implementation of the Multivariate Adaptive * * Regression Splines (MARS) methodology, first proposed by J.H. Friedman(1991)* * The Annals of Statistics, 19, 1 - 141. * * The program is a modified version of MARS (PolyMARS) as described in * * Kooperberg, C., Bose, S. and Stone C.J.(1997) ``Polychotomous Regression'', * * Journal of the American Statistical Association * * 92, 117 - 127. * * You are free to use this program, for non-commercial purposes only, * * under the condition that * * this note is not to be removed. * * * * The program is not formally maintained, but we are interested in hearing * * from people who have problems with it, although we may not be able to solve * * them. * * Email clk@fhcrc.org or martin@stat.washington.edu * * Charles Kooperberg and Martin O'Connor, May 20, 1997 * ******************************************************************************/ SHAR_EOF fi if test -f 'f2c.h' then echo shar: "will not over-write existing file 'f2c.h'" else cat << \SHAR_EOF > 'f2c.h' /* f2c.h -- Standard Fortran to C header file */ /** barf [ba:rf] 2. "He suggested using FORTRAN, and everybody barfed." - From The Shogakukan DICTIONARY OF NEW ENGLISH (Second edition) */ #ifndef F2C_INCLUDE #define F2C_INCLUDE typedef long int integer; typedef char *address; typedef short int shortint; typedef float real; typedef double doublereal; typedef struct { real r, i; } complex; typedef struct { doublereal r, i; } doublecomplex; typedef long int logical; typedef short int shortlogical; typedef char logical1; typedef char integer1; /* typedef long long longint; */ /* system-dependent */ #define TRUE_ (1) #define FALSE_ (0) /* Extern is for use with -E */ #ifndef Extern #define Extern extern #endif /* I/O stuff */ #ifdef f2c_i2 /* for -i2 */ typedef short flag; typedef short ftnlen; typedef short ftnint; #else typedef long flag; typedef long ftnlen; typedef long ftnint; #endif /*external read, write*/ typedef struct { flag cierr; ftnint ciunit; flag ciend; char *cifmt; ftnint cirec; } cilist; /*internal read, write*/ typedef struct { flag icierr; char *iciunit; flag iciend; char *icifmt; ftnint icirlen; ftnint icirnum; } icilist; /*open*/ typedef struct { flag oerr; ftnint ounit; char *ofnm; ftnlen ofnmlen; char *osta; char *oacc; char *ofm; ftnint orl; char *oblnk; } olist; /*close*/ typedef struct { flag cerr; ftnint cunit; char *csta; } cllist; /*rewind, backspace, endfile*/ typedef struct { flag aerr; ftnint aunit; } alist; /* inquire */ typedef struct { flag inerr; ftnint inunit; char *infile; ftnlen infilen; ftnint *inex; /*parameters in standard's order*/ ftnint *inopen; ftnint *innum; ftnint *innamed; char *inname; ftnlen innamlen; char *inacc; ftnlen inacclen; char *inseq; ftnlen inseqlen; char *indir; ftnlen indirlen; char *infmt; ftnlen infmtlen; char *inform; ftnint informlen; char *inunf; ftnlen inunflen; ftnint *inrecl; ftnint *innrec; char *inblank; ftnlen inblanklen; } inlist; #define VOID void union Multitype { /* for multiple entry points */ integer1 g; shortint h; integer i; /* longint j; */ real r; doublereal d; complex c; doublecomplex z; }; typedef union Multitype Multitype; typedef long Long; /* No longer used; formerly in Namelist */ struct Vardesc { /* for Namelist */ char *name; char *addr; ftnlen *dims; int type; }; typedef struct Vardesc Vardesc; struct Namelist { char *name; Vardesc **vars; int nvars; }; typedef struct Namelist Namelist; #define abs(x) ((x) >= 0 ? (x) : -(x)) #define dabs(x) (doublereal)abs(x) #define min(a,b) ((a) <= (b) ? (a) : (b)) #define max(a,b) ((a) >= (b) ? (a) : (b)) #define dmin(a,b) (doublereal)min(a,b) #define dmax(a,b) (doublereal)max(a,b) /* procedure parameter types for -A and -C++ */ #define F2C_proc_par_types 1 #ifdef __cplusplus typedef int /* Unknown procedure type */ (*U_fp)(...); typedef shortint (*J_fp)(...); typedef integer (*I_fp)(...); typedef real (*R_fp)(...); typedef doublereal (*D_fp)(...), (*E_fp)(...); typedef /* Complex */ VOID (*C_fp)(...); typedef /* Double Complex */ VOID (*Z_fp)(...); typedef logical (*L_fp)(...); typedef shortlogical (*K_fp)(...); typedef /* Character */ VOID (*H_fp)(...); typedef /* Subroutine */ int (*S_fp)(...); #else typedef int /* Unknown procedure type */ (*U_fp)(); typedef shortint (*J_fp)(); typedef integer (*I_fp)(); typedef real (*R_fp)(); typedef doublereal (*D_fp)(), (*E_fp)(); typedef /* Complex */ VOID (*C_fp)(); typedef /* Double Complex */ VOID (*Z_fp)(); typedef logical (*L_fp)(); typedef shortlogical (*K_fp)(); typedef /* Character */ VOID (*H_fp)(); typedef /* Subroutine */ int (*S_fp)(); #endif /* E_fp is for real functions when -R is not specified */ typedef VOID C_f; /* complex function */ typedef VOID H_f; /* character function */ typedef VOID Z_f; /* double complex function */ typedef doublereal E_f; /* real function with -R not specified */ /* undef any lower-case symbols that your C compiler predefines, e.g.: */ #ifndef Skip_f2c_Undefs #undef cray #undef gcos #undef mc68010 #undef mc68020 #undef mips #undef pdp11 #undef sgi #undef sparc #undef sun #undef sun2 #undef sun3 #undef sun4 #undef u370 #undef u3b #undef u3b2 #undef u3b5 #undef unix #undef vax #endif #endif SHAR_EOF fi if test -f 'fig1.eps' then echo shar: "will not over-write existing file 'fig1.eps'" else cat << \SHAR_EOF > 'fig1.eps' %!PS-Adobe-3.0 EPSF-3.0 %%Title: (S-PLUS Graphics) %%Creator: S-PLUS %%For: (& O'Connor) %%CreationDate: Tue May 6 15:38:29 1997 %%BoundingBox: 20 270 592 522 %%Pages: (atend) %%DocumentData: Clean7Bit %%DocumentNeededResources: (atend) %%EndComments %%BeginProlog 150 dict begin gsave /bd{bind def}def /PVer 3200 def % drawing commands /I{Coord SetPage 1 setlinecap 1 setlinejoin LineTypes{RastersPerPoint ScaleArray}forall /Helvetica findfont PointSize RastersPerPoint mul Cex mul scalefont setfont /Colors GColors def }bd /BWOp{ bw{currentrgbcolor add add 2.9 gt{1}{0}ifelse setgray}if exec bw{CurSc Sc}if }bd /IBu{}def /B{newpath}bd /E{{stroke}BWOp}bd /C{currentpoint Penup{newpath}{E}ifelse moveto}bd /F{closepath eofill}bd /M{moveto}bd /L{lineto}bd /Pu{load /XEQ exch def /Penup true def}bd /Pd{/XEQ{}def /Penup false def}bd /Px{currentdict /XEQ known{XEQ}if}bd /h{Penup{rmoveto Px}{rlineto}ifelse}bd /j{neg Penup{rmoveto Px}{rlineto}ifelse}bd /k{exch neg exch Penup{rmoveto Px}{rlineto}ifelse}bd /l{neg exch neg exch Penup{rmoveto Px}{rlineto}ifelse}bd /m{moveto Penup{Px}if}bd /x{0 Penup{rmoveto Px}{rlineto}ifelse}bd /y{0 exch Penup{rmoveto Px}{rlineto}ifelse}bd /w{neg x}bd /z{neg y}bd /a{B M dup y exch x neg y closepath}bd /b{a eofill}bd /c{a E}bd /A{PageBegin BackCol 0 ge {CurSc gsave 0 SC Page aload pop exch 4 -1 roll dup 5 1 roll sub 3 1 roll exch dup 4 1 roll sub 4 2 roll b grestore SC}if }bd /Ph{gsave Pch-x Pch-y rmoveto Show grestore}bd /Pc{Pch Ph}bd /T{/Adjust exch def gsave translate StringRot rotate 0 0 moveto dup stringwidth pop neg Adjust mul 0 rmoveto currentpoint translate TextShow grestore}bd /X{erasepage InPage{PageEnd}if}bd /Z{gsave showpage grestore PageEnd}bd /W{grestore end}bd % parameter setting commands /saveAt{Attr begin /Atlinewidth currentlinewidth def /Atfont currentfont def /Atdash currentdash 2 array astore def /AtSc CurSc def end}bd /restAt{Attr begin Atlinewidth Atfont Atdash AtSc end SC aload pop setdash setfont setlinewidth }bd /SC{dup dup 0 eq {pop BackIdx {BackCol dup 0 lt {pop 1 setgray} {Sc}ifelse} {/Colors BColor def 1 Sc}ifelse} {/Colors GColors def Sc}ifelse /CurSc exch def }bd /St{1 sub LineTypes dup 3 1 roll length Rem floor get 0 setdash}bd /Sw{abs 2 div RastersPerPoint mul setlinewidth SetClip}bd /Sp{dup 42 eq{pop IBu}{Pch exch 0 exch put SetPchSize}ifelse}bd /Sx{dup Cex div /Ratio exch def /Cex exch def currentfont Ratio scalefont setfont /Pch-x Pch-x Ratio mul def /Pch-y Pch-y Ratio mul def /Text-y Text-y Ratio mul def}bd /So{4 1 roll exch 4 -1 roll Plot astore pop SetClip}bd /Sg{4 1 roll exch 4 -1 roll Figure astore pop SetClip}bd /Sr{/StringRot exch def}bd /Sh{/CharRot exch def}bd /Sd{0 eq /ClipToPlot exch def SetClip}bd /Iv{/InPage false def/Clip 4 array def/Page 4 array def /Figure [0 0 1 1] def/Plot [0 0 1 1] def/ClipToPlot true def /Cex 1 def/Outline false def /Pch 1 string def/Pch-x 0 def/Pch-y 0 def/Text-y 0 def /LineTypes [ % in default units [] [1 2] [4 4] [8 4] [13 3] [16 2 2 2] [8 2 2 2] [1 13] [6 5] [12 4] ] def /Attr 5 dict def/CurSc 1 def/Penup false def/XEQ{}def }bd /Rem{2 copy div floor mul sub floor cvi}bd /RastersPerPoint{RastersPerInch 72 div}bd /ScaleArray{/Factor exch def /Array exch def 0 1 Array length 1 sub {dup Array exch get Factor mul Array 3 1 roll put}for}bd /Coord{Region aload pop /uy exch def /ux exch def /ly exch def /lx exch def uy ly sub ux lx sub Landscape{exch}if /Width exch def /Height exch def lx ly translate Landscape{90 rotate 0 Height neg translate}if 1 RastersPerPoint div dup scale}bd /SetPchSize{gsave newpath 0 0 moveto Pch false charpath flattenpath pathbbox exch 3 1 roll add 2 div neg /Pch-y exch def add 2 div neg /Pch-x exch def grestore}bd /Show{Outline{false charpath E}{{show}BWOp}ifelse}bd /SimpleShow{0 Text-y 2 div neg rmoveto Show}bd /FancyShow{ /RotDiff exch def /Cos RotDiff cos abs def /Sin RotDiff sin abs def { ( ) dup 0 4 -1 roll put dup stringwidth pop /CharWidth exch def Cos 0 eq{ Text-y Sin div }{ Sin 0 eq{ CharWidth Cos div }{ Text-y Sin div dup CharWidth Cos div dup 4 1 roll lt{exch pop}{pop}ifelse }ifelse }ifelse 2 div /CharDist exch def CharDist 0 translate 0 0 moveto gsave RotDiff rotate CharWidth 2 div neg Text-y 2 div neg rmoveto Show grestore CharDist 0 translate 0 0 moveto }forall }bd /TextShow{ CharRot StringRot sub dup 0 eq{pop SimpleShow}{FancyShow}ifelse }bd /BoxClip{/CLW currentlinewidth def 2{CLW add 4 1 roll}repeat 2{CLW sub 4 1 roll}repeat % initclip saveAt grestore gsave restAt newpath 2 index exch 2 index exch dup 6 index exch moveto 3{lineto}repeat closepath clip newpath}bd /Subregion{ 7 dict begin/A exch def/Uy exch def/Ux exch def/Ly exch def/Lx exch def Ux Lx sub A 0 get mul Lx add Uy Ly sub A 1 get mul Ly add Ux Lx sub A 2 get mul Lx add Uy Ly sub A 3 get mul Ly add end}bd /SetFigure{Page aload pop Figure Subregion}bd /SetPlot{SetFigure Plot Subregion}bd /SetClip{ClipToPlot{SetPlot}{SetFigure}ifelse BoxClip}bd /SetPage{0 0 Width Height Page astore RastersPerPoint ScaleArray}bd /PageBegin{save /PageContext exch def /InPage true def gsave}bd /PageEnd{grestore PageContext restore /InPage false def}bd /SImage{ dup /Bits exch def 8 exch idiv /PerByte exch def /Ht exch def /Wid exch def 1 Wid div 1 Ht div scale /saveSc CurSc def /Colors IColors def /curstr Wid PerByte add 1 sub PerByte idiv string def /MaskByte 16#ff Bits 8 sub bitshift def 1 1 Ht{ currentfile curstr readhexstring pop pop /CurPos 0 def /Left PerByte def 1 1 Wid{ Left PerByte eq{ /CurByte curstr CurPos get def /MaskMove -8 Bits add def /CurPos CurPos 1 add def }if CurByte MaskMove bitshift MaskByte and dup 0 gt{ Sc exch dup 3 1 roll B M 1 z 1 w 1 y F }{pop pop}ifelse Left 1 sub dup /Left exch def 0 le {/Left PerByte def} {/MaskMove MaskMove Bits add def}ifelse }for pop }for /Colors GColors def saveSc Sc }bd /Ic {} bd /Sc { dup dup 1 lt exch 0 ge and {1 exch sub setgray} % explicit gray level [0,1) {1 sub Colors dup 3 1 roll length Rem floor get dup type /arraytype eq {aload pop setrgbcolor} {setgray} ifelse } ifelse } bd /If {} bd /Sf {dup 0 lt /Outline exch def abs 1 sub Fonts dup 3 1 roll length Rem floor get findfont PointSize Cex mul RastersPerPoint mul scalefont dup setfont dup /FontMatrix get /Matrix exch def /FontBBox get aload pop Matrix transform 4 2 roll Matrix transform exch pop add /Text-y exch def pop SetPchSize} bd %%EndProlog %%BeginSetup % fixed controlling parameters /Landscape false def /Region [20.88 170.755 591.12 621.245] def /bw statusdict begin /processcolors where {pop processcolors} {1} ifelse end 1 eq def /RastersPerInch 300 def /PointSize 14 def /Fonts [ /Helvetica /Courier /Times-Roman /Helvetica-Oblique /Helvetica-Bold /Helvetica-BoldOblique /Courier-Oblique /Courier-Bold /Courier-BoldOblique /Times-Italic /Times-Bold /Times-BoldItalic /Symbol /AvantGarde-Book /AvantGarde-BookOblique /AvantGarde-Demi /AvantGarde-DemiOblique /Bookman-Demi /Bookman-DemiItalic /Bookman-Light /Bookman-LightItalic /Helvetica-Narrow /Helvetica-Narrow-Bold /Helvetica-Narrow-BoldOblique /Helvetica-Narrow-Oblique /NewCenturySchlbk-Roman /NewCenturySchlbk-Bold /NewCenturySchlbk-Italic /NewCenturySchlbk-BoldItalic /Palatino-Roman /Palatino-Bold /Palatino-Italic /Palatino-BoldItalic /ZapfChancery-MediumItalic /ZapfDingbats ] def /GColors [ 0 0.6 0.3 0.9 0.4 0.7 0.1 0.5 0.8 0.2 ] def /BackIdx true def /BackCol -1 def /IColors [ 0.933333 0.866667 0.8 0.733333 0.666667 0.6 0.533333 0.466667 0.4 0.333333 0.266667 0.2 0.133333 0.0666667 0 ] def /Colors GColors def /IBu {Pch 0 8#267 put SetPchSize} def /Bu {Pc} def % all initialization action here Iv I currentdict /Ic known {Ic} if currentdict /If known {If} if %%EndSetup %%Page: 1 1 %%PageOrientation: Portrait A 1 St 1 Sw 1 SC 0 Sr 42 Sp 1 Sx 0.123 0.937 0.0968354 0.922152 So 0 0.5 0 1 Sg 0 Sh 1 Sd %%IncludeResource: font Helvetica 1 Sf 544 656 m 13 9 k 530 665 m 13 10 k 516 675 m 13 8 k 502 683 m 13 9 k 488 692 m 12 10 k 475 702 m 12 8 k 462 710 m 12 10 k 449 720 m 13 8 k 435 728 m 11 9 k 423 737 m 12 9 k 410 746 m 11 9 k 398 755 m 13 8 k 384 763 m 11 9 k 372 772 m 11 8 k 360 780 m 12 9 k 347 789 m 11 8 k 335 797 m 11 8 k 323 805 m 11 9 k 311 814 m 11 7 k 299 821 m 11 9 k E 287 830 m 10 7 k 276 837 m 11 8 k 264 845 m 10 8 k 253 853 m 10 8 k 242 861 m 11 7 k 230 868 m 10 8 k 219 876 m 10 8 k 208 884 m 11 7 k 196 891 m 9 8 k 186 899 m 11 7 k 174 906 m 9 7 k 164 913 m 10 7 k 545 656 m 19 6 h 565 662 m 18 8 h 584 670 m 18 6 h 603 676 m 18 7 h 622 683 m 18 8 h 641 691 m 17 6 h 18 7 h 677 704 m 18 6 h E 696 710 m 17 7 h 714 717 m 17 7 h 732 724 m 17 6 h 750 730 m 17 7 h 768 737 m 17 7 h 786 744 m 17 6 h 804 750 m 16 7 h 821 757 m 17 6 h 839 763 m 16 7 h 856 770 m 16 6 h 873 776 m 16 6 h 890 782 m 16 7 h 907 789 m 16 6 h 924 795 m 16 6 h 941 801 m 16 6 h 958 807 m 15 7 h 974 814 m 15 5 h 990 819 m 16 6 h 1007 825 m 15 6 h 1023 831 m 15 6 h E 1039 837 m 16 6 h 16 6 h 1072 849 m 15 6 h 1088 855 m 15 6 h 550 673 m 18 8 l 551 673 m 12 10 j 536 683 m 18 8 l 537 683 m 12 9 j 569 680 m 17 6 l 570 680 m 12 9 j 522 692 m 18 8 l 523 692 m 12 8 j 554 692 m 17 8 l 555 692 m 13 11 j 588 688 m 17 7 l 589 688 m 12 11 j 508 703 m 18 10 l 509 703 m 12 10 j 541 702 m 18 9 l 542 702 m 12 10 j E 573 700 m 17 8 l 574 700 m 13 11 j 606 696 m 17 7 l 607 696 m 13 12 j 495 712 m 18 9 l 496 712 m 11 8 j 527 713 m 18 9 l 528 713 m 12 10 j 560 711 m 18 8 l 561 711 m 11 10 j 592 709 m 17 8 l 593 709 m 13 12 j 625 704 m 17 7 l 626 704 m 13 12 j 482 723 m 18 12 l 483 723 m 11 10 j 514 724 m 18 11 l 515 724 m 11 10 j 546 723 m 18 9 l 547 723 m 12 11 j E 579 721 m 18 9 l 580 721 m 11 12 j 611 717 m 17 8 l 612 717 m 12 12 j 644 711 m 17 6 l 645 709 m 12 11 j 468 732 m 17 11 l 469 732 m 12 8 j 501 734 m 18 10 l 502 734 m 11 9 j 533 734 m 18 9 l 534 734 m 11 10 j 565 733 m 18 9 l 566 733 m 12 11 j 597 730 m 17 8 l 598 730 m 12 12 j 630 726 m 17 8 l 631 726 m 12 13 j 662 719 m 17 8 l 663 717 m 12 12 j E 455 742 m 18 13 l 456 742 m 12 9 j 487 744 m 18 11 l 488 744 m 12 9 j 520 745 m 18 10 l 520 745 m 12 10 j 552 745 m 18 10 l 553 745 m 11 11 j 584 743 m 18 9 l 585 743 m 11 12 j 616 739 m 17 8 l 617 739 m 12 12 j 648 733 m 16 6 l 649 733 m 12 12 j 681 727 m 18 8 l 682 725 m 12 14 j 442 751 m 17 13 l 443 751 m 11 8 j 474 755 m 18 12 l 475 755 m 11 10 j E 506 757 m 18 12 l 507 757 m 12 11 j 537 757 m 17 11 l 538 757 m 13 11 j 570 756 m 17 10 l 571 756 m 12 12 j 602 753 m 17 9 l 603 753 m 12 12 j 635 748 m 18 8 l 636 746 m 11 12 j 667 742 m 17 8 l 668 740 m 12 13 j 699 734 m 17 7 l 700 732 m 12 14 j 430 761 m 18 14 l 431 761 m 10 9 j 461 765 m 18 13 l 462 765 m 11 9 j 493 768 m 18 12 l 494 768 m 11 10 j E 524 769 m 17 11 l 525 769 m 12 11 j 556 768 m 18 10 l 557 768 m 12 11 j 589 766 m 18 9 l 589 766 m 12 11 j 621 762 m 18 8 l 622 762 m 12 12 j 653 757 m 17 9 l 654 755 m 12 12 j 685 750 m 17 8 l 686 748 m 12 13 j 717 741 m 17 7 l 718 739 m 12 14 j 416 770 m 17 14 l 417 770 m 12 8 j 448 776 m 17 14 l 449 776 m 11 10 j 480 779 m 18 13 l 481 779 m 11 10 j E 511 780 m 17 11 l 512 780 m 11 10 j 543 781 m 18 11 l 544 781 m 11 11 j 575 779 m 18 10 l 576 779 m 12 11 j 606 777 m 17 10 l 607 777 m 13 13 j 639 772 m 17 9 l 640 770 m 12 12 j 671 766 m 17 8 l 672 764 m 12 13 j 703 758 m 17 8 l 704 756 m 12 14 j 735 748 m 17 7 l 736 746 m 12 15 j 404 779 m 18 15 l 405 779 m 11 8 j 435 785 m 18 14 l 436 785 m 11 8 j E 467 790 m 18 13 l 468 790 m 11 11 j 498 793 m 17 14 l 499 793 m 11 12 j 530 794 m 18 13 l 531 794 m 11 12 j 561 794 m 17 12 l 562 794 m 12 13 j 593 791 m 17 11 l 594 789 m 12 11 j 625 787 m 18 9 l 626 785 m 12 12 j 658 782 m 18 9 l 658 780 m 12 13 j 689 775 m 17 9 l 690 773 m 12 14 j 721 766 m 17 8 l 722 764 m 12 15 j 753 756 m 17 8 l 754 754 m 12 16 j E 391 789 m 17 16 l 392 789 m 11 9 j 422 796 m 17 16 l 423 796 m 11 10 j 453 800 m 17 14 l 454 800 m 12 9 j 485 804 m 17 13 l 485 804 m 12 10 j 517 806 m 18 12 l 518 806 m 11 11 j 548 807 m 17 12 l 549 807 m 11 11 j 580 805 m 18 10 l 581 803 m 11 11 j 611 802 m 17 10 l 612 800 m 12 12 j 643 797 m 17 9 l 644 796 m 13 13 j 675 792 m 17 9 l 676 790 m 12 14 j E 707 784 m 17 8 l 708 782 m 12 15 j 739 775 m 17 9 l 740 773 m 12 16 j 771 763 m 17 7 l 772 762 m 12 18 j 379 797 m 17 16 l 380 797 m 10 7 j 410 805 m 18 15 l 411 805 m 10 8 j 441 812 m 18 15 l 442 812 m 10 11 j 472 815 m 18 14 l 473 815 m 11 10 j 503 818 m 18 13 l 504 818 m 12 11 j 535 820 m 17 13 l 536 820 m 11 11 j 567 819 m 18 11 l 568 817 m 11 11 j E 598 817 m 17 11 l 599 815 m 11 12 j 630 814 m 18 11 l 631 812 m 11 14 j 661 808 m 17 10 l 662 806 m 13 13 j 693 801 m 17 8 l 694 799 m 12 14 j 725 793 m 17 9 l 726 791 m 12 15 j 757 782 m 17 7 l 758 780 m 12 16 j 789 770 m 17 7 l 790 768 m 12 17 j 366 807 m 17 16 l 367 807 m 11 9 j 398 815 m 18 17 l 399 815 m 10 9 j 428 822 m 17 16 l 429 822 m 11 9 j E 459 828 m 17 15 l 460 828 m 11 12 j 490 831 m 17 15 l 491 831 m 11 12 j 521 833 m 17 14 l 522 833 m 12 11 j 553 833 m 17 12 l 554 833 m 12 12 j 585 832 m 17 12 l 586 831 m 11 13 j 616 830 m 17 12 l 617 828 m 12 14 j 648 825 m 17 11 l 649 823 m 11 14 j 679 818 m 17 9 l 680 816 m 13 14 j 711 811 m 17 9 l 712 809 m 12 15 j 743 801 m 17 8 l 744 799 m 12 16 j E 775 790 m 17 8 l 776 788 m 12 17 j 806 778 m 16 8 l 807 776 m 12 18 j 354 815 m 17 18 l 355 815 m 10 7 j 384 825 m 17 16 l 385 825 m 12 9 j 416 832 m 17 16 l 416 832 m 11 9 j 447 839 m 18 16 l 448 839 m 10 10 j 477 843 m 17 14 l 478 843 m 11 12 j 508 846 m 17 15 l 509 846 m 11 12 j 539 848 m 17 14 l 540 846 m 12 12 j 572 847 m 18 13 l 572 845 m 12 12 j E 603 845 m 17 12 l 604 843 m 11 12 j 634 841 m 17 10 l 635 839 m 12 13 j 666 835 m 17 9 l 667 833 m 11 14 j 697 829 m 17 10 l 698 827 m 12 15 j 729 820 m 17 8 l 730 818 m 12 16 j 761 810 m 17 9 l 762 808 m 12 17 j 792 798 m 16 7 l 793 797 m 12 18 j 824 785 m 17 7 l 825 783 m 12 19 j 342 825 m 17 19 l 343 825 m 10 9 j 372 834 m 17 18 l 373 834 m 10 8 j E 403 843 m 18 17 l 404 843 m 11 10 j 433 850 m 17 16 l 434 850 m 12 10 j 465 855 m 17 15 l 466 855 m 10 10 j 496 859 m 18 15 l 497 857 m 10 10 j 526 861 m 17 14 l 527 861 m 11 12 j 558 861 m 18 12 l 559 859 m 12 11 j 589 860 m 17 12 l 590 858 m 12 12 j 621 857 m 17 11 l 622 855 m 11 13 j 652 852 m 17 10 l 653 850 m 12 14 j 684 847 m 17 11 l 685 845 m 11 15 j E 715 839 m 17 9 l 716 837 m 12 16 j 746 830 m 16 9 l 747 828 m 13 17 j 779 819 m 17 8 l 779 817 m 12 18 j 810 806 m 17 8 l 811 804 m 12 18 j 841 792 m 16 7 l 842 790 m 12 19 j 330 833 m 18 19 l 331 833 m 10 7 j 360 845 m 17 19 l 361 845 m 10 10 j 390 853 m 17 18 l 391 853 m 11 9 j 421 861 m 17 16 l 422 861 m 11 10 j 451 867 m 17 16 l 452 867 m 12 11 j E 483 871 m 17 15 l 484 871 m 11 11 j 514 874 m 17 14 l 515 872 m 10 10 j 545 876 m 18 14 l 546 874 m 11 12 j 576 875 m 17 13 l 577 873 m 12 12 j 607 873 m 17 12 l 608 871 m 12 13 j 639 869 m 17 11 l 640 867 m 11 14 j 670 865 m 17 12 l 671 863 m 12 15 j 701 858 m 16 10 l 702 856 m 12 16 j 733 849 m 17 9 l 734 848 m 11 17 j 764 839 m 17 8 l 765 837 m 13 17 j E 796 828 m 17 8 l 796 826 m 13 19 j 827 814 m 16 8 l 828 812 m 12 19 j 858 799 m 16 7 l 859 797 m 12 20 j 318 842 m 17 20 l 319 842 m 11 8 j 348 854 m 17 20 l 349 854 m 10 8 j 378 864 m 17 18 l 379 864 m 10 10 j 409 872 m 18 17 l 410 872 m 10 10 j 439 879 m 17 16 l 440 879 m 11 11 j 469 884 m 17 16 l 470 884 m 12 11 j 501 887 m 17 15 l 502 887 m 11 11 j E 532 890 m 17 15 l 533 888 m 11 11 j 563 890 m 17 13 l 564 888 m 11 12 j 594 889 m 17 13 l 595 887 m 11 13 j 625 886 m 17 12 l 626 884 m 12 14 j 657 882 m 17 12 l 658 880 m 11 14 j 688 876 m 17 10 l 689 874 m 11 15 j 719 868 m 17 9 l 720 866 m 12 16 j 750 859 m 16 9 l 751 857 m 12 17 j 781 849 m 16 9 l 782 847 m 13 18 j 813 836 m 17 8 l 814 834 m 12 20 j E 845 822 m 17 8 l 846 820 m 11 20 j 875 806 m 16 7 l 876 804 m 12 21 j 306 851 m 17 20 l 307 851 m 10 8 j 336 864 m 17 21 l 337 864 m 10 9 j 365 874 m 16 19 l 366 874 m 11 9 j 396 884 m 17 19 l 397 884 m 11 11 j 427 891 m 17 18 l 428 891 m 10 10 j 457 897 m 17 16 l 458 897 m 10 11 j 487 901 m 17 16 l 488 900 m 12 12 j 519 903 m 17 15 l 520 903 m 11 11 j E 550 905 m 17 14 l 551 903 m 11 12 j 581 904 m 17 13 l 582 902 m 11 12 j 612 902 m 17 12 l 613 901 m 11 14 j 643 899 m 17 12 l 644 897 m 12 14 j 675 894 m 17 11 l 675 892 m 12 15 j 706 887 m 17 10 l 707 885 m 11 16 j 737 879 m 17 10 l 738 877 m 11 17 j 768 869 m 17 9 l 769 867 m 11 18 j 799 858 m 17 9 l 800 856 m 13 19 j 831 845 m 17 8 l 831 843 m 13 20 j E 862 830 m 16 8 l 863 828 m 11 21 j 892 813 m 16 7 l 893 811 m 12 21 j 295 860 m 17 22 l 295 860 m 10 8 j 324 873 m 17 21 l 325 873 m 10 8 j 354 884 m 17 19 l 355 884 m 9 9 j 383 894 m 17 19 l 384 894 m 11 10 j 414 902 m 17 18 l 415 902 m 11 10 j 444 909 m 16 17 l 445 909 m 11 11 j 475 914 m 17 16 l 476 914 m 10 11 j 505 918 m 17 16 l 506 916 m 12 12 j E 537 919 m 17 15 l 537 918 m 12 12 j 568 920 m 17 14 l 569 919 m 11 14 j 598 919 m 16 14 l 599 917 m 12 14 j 629 916 m 16 13 l 630 914 m 12 14 j 660 912 m 16 12 l 661 910 m 13 15 j 692 906 m 17 11 l 693 904 m 12 16 j 723 899 m 16 11 l 724 897 m 12 17 j 754 890 m 16 10 l 755 888 m 12 18 j 785 879 m 16 9 l 786 877 m 12 18 j 816 866 m 16 7 l 817 865 m 13 19 j E 848 853 m 17 7 l 848 851 m 13 20 j 879 837 m 16 7 l 880 835 m 11 21 j 909 820 m 16 7 l 910 818 m 12 22 j 282 868 m 16 22 l 283 868 m 11 7 j 312 883 m 17 22 l 313 883 m 10 9 j 342 895 m 17 21 l 343 895 m 10 10 j 371 905 m 16 20 l 372 905 m 10 10 j 401 914 m 17 19 l 402 914 m 11 11 j 432 921 m 17 18 l 433 921 m 10 10 j 462 927 m 17 16 l 463 927 m 11 11 j E 493 932 m 17 16 l 494 930 m 10 11 j 523 934 m 17 15 l 524 932 m 12 12 j 554 936 m 17 16 l 555 934 m 12 13 j 585 935 m 16 14 l 586 933 m 11 14 j 616 933 m 17 14 l 617 931 m 11 14 j 647 930 m 17 13 l 648 928 m 11 15 j 678 924 m 17 11 l 679 922 m 12 15 j 710 918 m 17 11 l 710 916 m 12 16 j 741 910 m 17 10 l 742 908 m 11 17 j 772 901 m 17 10 l 773 899 m 11 19 j E 803 889 m 17 9 l 804 887 m 11 20 j 833 876 m 16 9 l 834 874 m 13 20 j 865 861 m 17 8 l 865 859 m 13 21 j 896 845 m 16 8 l 897 843 m 11 22 j 926 827 m 16 7 l 927 825 m 12 23 j 271 877 m 16 23 l 272 877 m 9 8 j 300 891 m 17 22 l 301 891 m 11 7 j 330 904 m 17 20 l 330 904 m 11 8 j 360 916 m 17 20 l 361 916 m 9 10 j 389 925 m 17 19 l 390 925 m 10 10 j E 419 934 m 17 19 l 420 934 m 11 11 j 450 940 m 17 18 l 451 938 m 10 10 j 480 945 m 17 16 l 481 945 m 11 11 j 511 949 m 17 16 l 512 947 m 10 12 j 541 951 m 17 16 l 542 949 m 12 13 j 572 951 m 17 15 l 573 949 m 11 13 j 603 950 m 17 14 l 604 948 m 11 14 j 634 947 m 17 13 l 635 945 m 11 14 j 665 943 m 17 12 l 666 941 m 11 16 j 696 937 m 17 12 l 697 936 m 12 17 j E 727 930 m 17 11 l 727 928 m 13 17 j 758 921 m 16 10 l 759 919 m 12 18 j 789 911 m 16 10 l 790 909 m 12 19 j 820 899 m 16 9 l 821 897 m 11 20 j 851 885 m 17 8 l 852 884 m 12 22 j 882 869 m 17 7 l 883 867 m 12 21 j 913 852 m 16 7 l 914 850 m 11 22 j 943 833 m 16 6 l 944 831 m 12 23 j 260 885 m 17 23 l 261 885 m 9 7 j 289 901 m 17 23 l 290 901 m 9 9 j E 318 915 m 17 23 l 319 915 m 10 10 j 347 926 m 17 21 l 348 926 m 11 9 j 377 937 m 16 20 l 378 937 m 10 10 j 407 946 m 17 20 l 408 946 m 10 10 j 437 953 m 17 18 l 438 953 m 11 11 j 468 958 m 17 16 l 468 956 m 11 10 j 498 963 m 17 16 l 499 961 m 11 11 j 528 965 m 16 15 l 529 963 m 11 11 j 559 966 m 17 14 l 560 964 m 12 12 j 589 966 m 16 14 l 590 964 m 12 13 j E 621 964 m 17 13 l 622 962 m 11 14 j 652 960 m 17 12 l 653 958 m 11 14 j 682 955 m 16 11 l 683 954 m 12 16 j 713 950 m 16 12 l 714 948 m 12 17 j 744 941 m 17 10 l 745 939 m 12 17 j 776 932 m 17 10 l 777 930 m 11 18 j 806 921 m 16 9 l 807 919 m 12 19 j 837 908 m 16 8 l 838 906 m 12 20 j 868 894 m 16 8 l 869 892 m 12 22 j 899 878 m 16 8 l 900 876 m 12 23 j E 929 860 m 15 8 l 930 858 m 12 24 j 959 840 m 15 7 l 960 838 m 12 24 j 248 894 m 16 25 l 249 894 m 10 8 j 277 910 m 16 24 l 278 910 m 10 9 j 306 924 m 16 23 l 307 924 m 10 8 j 335 937 m 16 21 l 336 937 m 11 10 j 364 948 m 16 21 l 365 948 m 11 10 j 395 957 m 17 19 l 396 957 m 10 10 j 425 966 m 17 19 l 426 964 m 10 10 j 455 971 m 17 17 l 456 970 m 11 11 j E 485 976 m 17 16 l 486 974 m 11 10 j 516 980 m 17 16 l 517 978 m 10 12 j 546 982 m 17 16 l 547 980 m 11 13 j 576 982 m 16 15 l 577 980 m 12 13 j 607 981 m 17 14 l 608 979 m 12 14 j 639 978 m 17 13 l 640 976 m 11 15 j 669 974 m 16 13 l 670 972 m 11 16 j 700 969 m 17 13 l 701 967 m 11 16 j 731 962 m 17 11 l 732 960 m 12 18 j 762 954 m 17 12 l 762 952 m 13 19 j E 793 943 m 16 10 l 794 941 m 11 19 j 823 932 m 16 10 l 824 930 m 12 21 j 854 919 m 16 10 l 855 917 m 12 22 j 884 903 m 15 8 l 885 901 m 13 22 j 916 886 m 16 7 l 917 884 m 11 23 j 946 867 m 16 7 l 947 866 m 12 25 j 976 848 m 16 8 l 977 846 m 12 26 j 237 901 m 16 24 l 238 901 m 9 6 j 265 919 m 16 24 l 266 919 m 10 8 j 295 934 m 17 23 l 295 934 m 10 9 j E 324 948 m 17 23 l 325 948 m 9 10 j 353 959 m 17 21 l 354 959 m 10 9 j 382 970 m 17 21 l 383 970 m 11 11 j 413 978 m 17 20 l 414 978 m 10 10 j 442 985 m 16 18 l 443 985 m 11 12 j 472 990 m 16 18 l 473 988 m 12 11 j 502 994 m 16 17 l 503 992 m 12 11 j 533 997 m 16 15 l 534 995 m 11 12 j 564 998 m 17 15 l 565 996 m 10 13 j 594 997 m 17 14 l 595 995 m 11 13 j E 625 995 m 17 13 l 626 993 m 12 14 j 656 992 m 16 13 l 657 990 m 11 15 j 687 988 m 17 13 l 688 986 m 11 16 j 717 981 m 16 11 l 718 979 m 12 16 j 748 973 m 16 10 l 749 971 m 12 17 j 779 965 m 17 11 l 779 963 m 13 19 j 810 954 m 16 10 l 811 953 m 11 20 j 841 941 m 17 8 l 842 939 m 11 20 j 871 928 m 16 9 l 872 926 m 11 22 j 901 912 m 16 8 l 902 910 m 13 23 j E 932 894 m 15 8 l 933 892 m 12 24 j 962 875 m 15 8 l 963 873 m 12 24 j 992 854 m 15 6 l 993 852 m 13 26 j 226 910 m 17 26 l 226 910 m 10 8 j 254 928 m 16 26 l 255 928 m 9 9 j 283 944 m 17 25 l 284 944 m 10 9 j 312 958 m 17 23 l 312 958 m 11 9 j 341 971 m 16 22 l 342 971 m 10 11 j 370 981 m 16 21 l 371 981 m 11 10 j 400 990 m 17 19 l 401 990 m 11 10 j E 430 998 m 16 19 l 431 996 m 10 10 j 460 1004 m 17 18 l 461 1002 m 10 11 j 490 1008 m 17 16 l 491 1006 m 11 11 j 520 1011 m 17 16 l 521 1009 m 11 11 j 551 1013 m 17 15 l 552 1011 m 11 12 j 581 1013 m 16 14 l 582 1011 m 11 13 j 612 1012 m 17 14 l 613 1010 m 11 14 j 642 1009 m 16 13 l 643 1007 m 12 14 j 674 1006 m 17 13 l 675 1004 m 11 16 j 704 1001 m 16 13 l 705 999 m 11 17 j E 735 994 m 17 12 l 736 992 m 11 18 j 765 986 m 16 12 l 766 984 m 12 18 j 796 976 m 17 10 l 797 974 m 12 19 j 827 965 m 16 10 l 828 963 m 12 21 j 858 952 m 16 10 l 859 950 m 11 21 j 888 937 m 16 8 l 889 936 m 11 23 j 918 920 m 16 7 l 919 919 m 12 24 j 949 902 m 16 7 l 950 901 m 12 25 j 979 883 m 16 8 l 980 881 m 12 26 j 1009 861 m 16 7 l 1010 859 m 12 27 j E 214 919 m 16 27 l 215 919 m 10 8 j 243 936 m 17 25 l 243 936 m 10 7 j 271 954 m 16 25 l 272 954 m 10 9 j 300 969 m 16 24 l 301 969 m 10 10 j 330 982 m 18 23 l 330 982 m 10 10 j 359 993 m 17 22 l 360 991 m 9 9 j 388 1003 m 17 21 l 389 1003 m 10 11 j 417 1011 m 16 20 l 418 1009 m 11 10 j 448 1018 m 17 19 l 449 1016 m 10 11 j 478 1023 m 17 18 l 479 1021 m 10 12 j E 507 1026 m 16 17 l 508 1024 m 12 12 j 537 1028 m 16 15 l 538 1026 m 12 12 j 569 1029 m 17 15 l 570 1027 m 10 13 j 599 1028 m 17 14 l 600 1026 m 11 13 j 629 1026 m 16 13 l 630 1024 m 11 14 j 660 1023 m 17 13 l 661 1022 m 12 16 j 691 1020 m 16 14 l 692 1018 m 11 16 j 721 1013 m 16 11 l 722 1011 m 12 16 j 752 1006 m 16 11 l 753 1005 m 11 18 j 782 998 m 16 11 l 783 996 m 13 19 j E 813 988 m 16 11 l 814 986 m 12 20 j 844 975 m 16 9 l 845 973 m 12 20 j 874 962 m 15 9 l 875 960 m 12 22 j 905 947 m 16 9 l 906 945 m 11 24 j 935 930 m 16 9 l 936 928 m 13 25 j 966 911 m 16 8 l 967 909 m 12 25 j 995 890 m 15 7 l 996 888 m 13 26 j 1025 867 m 15 6 l 1026 866 m 12 28 j 204 926 m 16 26 l 205 926 m 8 7 j 231 945 m 16 26 l 232 945 m 10 8 j E 260 963 m 17 26 l 261 963 m 9 9 j 289 978 m 17 24 l 290 978 m 9 8 j 317 992 m 16 22 l 318 992 m 11 9 j 347 1005 m 17 22 l 347 1005 m 11 11 j 376 1015 m 16 21 l 377 1015 m 10 10 j 405 1023 m 16 19 l 406 1023 m 10 10 j 435 1030 m 17 18 l 436 1030 m 11 10 j 465 1036 m 16 17 l 466 1036 m 11 12 j 495 1041 m 16 18 l 496 1039 m 10 12 j 525 1043 m 17 16 l 526 1041 m 11 12 j E 555 1044 m 17 15 l 556 1042 m 12 12 j 586 1044 m 16 14 l 587 1042 m 11 13 j 616 1043 m 16 14 l 617 1041 m 11 14 j 647 1041 m 17 14 l 648 1040 m 11 16 j 677 1038 m 16 14 l 678 1036 m 12 15 j 708 1032 m 16 11 l 709 1030 m 11 16 j 739 1026 m 17 12 l 740 1024 m 11 17 j 769 1019 m 16 12 l 770 1017 m 11 18 j 799 1009 m 16 10 l 800 1007 m 13 19 j 831 998 m 17 10 l 831 996 m 12 20 j E 861 986 m 16 10 l 862 984 m 11 21 j 891 971 m 16 8 l 892 970 m 12 22 j 921 956 m 15 8 l 922 954 m 12 23 j 952 938 m 16 7 l 952 936 m 13 24 j 982 919 m 15 8 l 983 917 m 12 26 j 1011 897 m 15 7 l 1012 895 m 13 27 j 1041 874 m 15 7 l 1042 872 m 13 28 j 192 935 m 16 28 l 193 935 m 10 8 j 220 954 m 15 27 l 221 954 m 9 8 j 248 972 m 16 26 l 249 972 m 10 8 j E 277 988 m 16 24 l 278 988 m 10 9 j 306 1003 m 16 24 l 307 1003 m 9 10 j 334 1016 m 16 23 l 335 1016 m 11 10 j 364 1026 m 17 20 l 364 1026 m 11 10 j 393 1036 m 16 20 l 394 1036 m 10 11 j 423 1043 m 17 19 l 424 1041 m 10 10 j 452 1050 m 16 19 l 453 1048 m 11 11 j 483 1055 m 17 18 l 484 1053 m 10 12 j 513 1058 m 17 17 l 514 1056 m 10 12 j 542 1059 m 16 15 l 543 1058 m 11 13 j E 572 1060 m 16 15 l 573 1058 m 12 13 j 604 1059 m 17 14 l 605 1058 m 10 14 j 634 1058 m 17 14 l 635 1057 m 11 15 j 664 1055 m 16 13 l 665 1053 m 11 14 j 694 1051 m 16 12 l 695 1049 m 12 16 j 726 1045 m 17 12 l 727 1043 m 11 16 j 756 1039 m 16 12 l 757 1037 m 11 17 j 786 1030 m 16 10 l 787 1028 m 11 18 j 816 1021 m 16 11 l 817 1019 m 13 20 j 848 1009 m 17 10 l 848 1007 m 12 20 j E 878 996 m 16 9 l 879 994 m 11 22 j 908 982 m 16 10 l 909 980 m 11 23 j 938 965 m 16 8 l 939 963 m 12 24 j 969 947 m 17 8 l 969 945 m 13 26 j 998 926 m 15 7 l 999 924 m 12 26 j 1028 904 m 16 7 l 1029 902 m 12 27 j 1057 881 m 15 7 l 1058 879 m 13 30 j 181 942 m 15 28 l 182 942 m 9 6 j 209 963 m 16 27 l 210 963 m 9 8 j 237 982 m 16 27 l 238 982 m 9 9 j E 265 999 m 16 26 l 266 999 m 10 10 j 294 1014 m 16 25 l 295 1014 m 10 10 j 323 1027 m 16 23 l 324 1027 m 9 9 j 352 1039 m 17 22 l 353 1037 m 10 10 j 382 1048 m 18 21 l 382 1046 m 10 9 j 411 1057 m 17 20 l 412 1055 m 10 11 j 440 1063 m 16 19 l 441 1061 m 10 10 j 470 1068 m 17 16 l 471 1066 m 11 10 j 500 1072 m 16 15 l 501 1070 m 11 12 j 530 1075 m 16 16 l 531 1073 m 10 13 j E 560 1076 m 17 16 l 561 1074 m 11 13 j 590 1076 m 17 15 l 591 1074 m 12 14 j 621 1075 m 16 15 l 622 1073 m 11 14 j 651 1072 m 16 13 l 652 1070 m 11 14 j 681 1069 m 16 13 l 682 1067 m 11 15 j 711 1064 m 16 12 l 712 1062 m 13 16 j 743 1058 m 16 12 l 744 1057 m 11 17 j 773 1051 m 16 11 l 774 1049 m 11 18 j 803 1042 m 16 11 l 804 1041 m 11 19 j 833 1032 m 16 10 l 834 1030 m 13 20 j E 864 1021 m 16 11 l 865 1019 m 12 22 j 895 1006 m 16 9 l 896 1005 m 11 22 j 924 991 m 15 8 l 925 989 m 12 23 j 954 974 m 15 8 l 955 972 m 13 24 j 985 955 m 16 7 l 986 954 m 12 27 j 1014 935 m 15 8 l 1015 933 m 13 28 j 1044 912 m 15 8 l 1045 910 m 12 28 j 1073 887 m 15 6 l 1074 885 m 13 29 j 171 951 m 16 30 l 172 951 m 8 8 j 198 971 m 16 28 l 199 971 m 10 7 j E 226 991 m 16 27 l 226 991 m 10 8 j 254 1008 m 16 25 l 255 1008 m 9 8 j 282 1024 m 16 24 l 283 1024 m 10 9 j 312 1039 m 17 24 l 312 1039 m 10 10 j 340 1050 m 16 22 l 341 1050 m 10 10 j 369 1060 m 16 20 l 370 1060 m 11 10 j 399 1069 m 17 20 l 399 1069 m 11 11 j 428 1076 m 16 18 l 429 1075 m 10 11 j 457 1081 m 16 17 l 458 1081 m 11 11 j 487 1086 m 16 17 l 488 1084 m 11 11 j E 518 1089 m 17 16 l 519 1087 m 10 11 j 547 1091 m 16 15 l 548 1089 m 11 13 j 577 1091 m 16 15 l 578 1089 m 11 13 j 607 1091 m 16 15 l 608 1089 m 12 13 j 638 1089 m 16 13 l 639 1087 m 11 14 j 668 1086 m 16 13 l 669 1084 m 11 14 j 698 1082 m 16 12 l 699 1080 m 11 15 j 729 1076 m 17 11 l 730 1075 m 12 16 j 760 1071 m 16 12 l 761 1069 m 11 17 j 790 1062 m 16 10 l 791 1060 m 11 17 j E 820 1054 m 16 11 l 821 1052 m 11 19 j 850 1043 m 16 10 l 851 1041 m 12 19 j 881 1031 m 16 9 l 882 1029 m 12 22 j 911 1017 m 15 10 l 912 1015 m 11 23 j 941 1001 m 16 9 l 942 999 m 11 24 j 971 984 m 16 9 l 972 982 m 13 26 j 1001 964 m 15 8 l 1002 962 m 12 26 j 1031 942 m 16 7 l 1032 940 m 12 27 j 1060 919 m 15 7 l 1061 917 m 12 29 j 1089 893 m 15 6 l 1090 891 m 13 29 j E 188 980 m 16 28 l 189 980 m 8 8 j 215 1000 m 16 28 l 216 1000 m 9 8 j 243 1019 m 17 27 l 243 1019 m 10 10 j 271 1035 m 16 26 l 272 1035 m 9 10 j 299 1049 m 16 24 l 300 1049 m 11 9 j 329 1061 m 17 21 l 330 1061 m 9 9 j 357 1073 m 16 22 l 358 1073 m 10 11 j 386 1081 m 16 20 l 387 1081 m 11 10 j 416 1089 m 17 19 l 416 1089 m 11 11 j 445 1094 m 16 17 l 446 1093 m 10 11 j E 475 1099 m 17 16 l 476 1097 m 10 10 j 504 1103 m 16 15 l 505 1101 m 12 11 j 535 1105 m 16 15 l 536 1103 m 10 11 j 565 1106 m 17 14 l 566 1104 m 10 12 j 595 1106 m 17 14 l 596 1104 m 10 12 j 625 1105 m 17 13 l 626 1103 m 11 13 j 655 1102 m 16 12 l 656 1100 m 11 13 j 686 1099 m 17 12 l 687 1097 m 10 14 j 716 1094 m 17 11 l 717 1093 m 11 16 j 746 1089 m 16 12 l 747 1087 m 12 15 j E 777 1082 m 16 10 l 778 1080 m 11 17 j 807 1075 m 16 12 l 808 1073 m 11 18 j 837 1065 m 16 10 l 838 1063 m 11 19 j 867 1054 m 16 10 l 868 1052 m 12 20 j 898 1041 m 16 9 l 899 1040 m 11 22 j 928 1027 m 16 9 l 929 1025 m 11 23 j 957 1011 m 15 9 l 958 1009 m 12 24 j 987 992 m 15 7 l 988 990 m 13 25 j 1017 972 m 15 7 l 1018 971 m 13 28 j 1047 950 m 15 8 l 1048 948 m 12 29 j E 1075 926 m 14 7 l 1076 924 m 13 30 j 204 1009 m 15 28 l 205 1009 m 9 8 j 231 1028 m 15 27 l 232 1028 m 10 8 j 260 1045 m 17 25 l 261 1045 m 9 9 j 288 1060 m 16 24 l 289 1060 m 9 9 j 316 1073 m 16 23 l 317 1073 m 11 11 j 346 1084 m 16 22 l 347 1084 m 9 9 j 375 1093 m 17 19 l 376 1092 m 9 10 j 403 1101 m 16 19 l 404 1101 m 11 10 j 433 1108 m 17 18 l 434 1106 m 10 11 j E 463 1112 m 17 16 l 464 1110 m 10 10 j 492 1116 m 16 15 l 493 1114 m 10 10 j 522 1119 m 17 15 l 523 1117 m 11 11 j 552 1120 m 16 14 l 553 1118 m 11 11 j 582 1120 m 16 13 l 583 1118 m 11 11 j 612 1120 m 16 13 l 613 1118 m 11 12 j 642 1118 m 16 12 l 643 1116 m 11 13 j 673 1115 m 17 12 l 674 1113 m 11 13 j 703 1111 m 16 11 l 704 1110 m 11 15 j 733 1107 m 16 12 l 734 1105 m 11 15 j E 763 1101 m 16 11 l 764 1099 m 12 16 j 794 1094 m 16 11 l 795 1093 m 11 17 j 824 1086 m 16 10 l 825 1084 m 11 18 j 854 1076 m 16 10 l 855 1075 m 11 20 j 884 1065 m 16 10 l 885 1063 m 12 21 j 914 1052 m 15 10 l 915 1050 m 12 22 j 944 1037 m 15 9 l 945 1035 m 11 23 j 974 1021 m 16 9 l 975 1019 m 12 26 j 1004 1002 m 16 9 l 1004 1000 m 13 27 j 1033 981 m 15 8 l 1034 979 m 13 28 j E 1062 958 m 14 7 l 1063 956 m 12 29 j 221 1038 m 16 28 l 222 1038 m 8 9 j 248 1056 m 16 27 l 249 1056 m 10 10 j 277 1071 m 16 25 l 278 1071 m 9 10 j 305 1084 m 16 23 l 306 1084 m 9 9 j 333 1096 m 16 22 l 334 1096 m 11 10 j 363 1106 m 16 21 l 364 1106 m 10 11 j 392 1113 m 16 19 l 393 1111 m 9 9 j 421 1120 m 17 18 l 422 1120 m 11 10 j 451 1126 m 17 17 l 451 1124 m 11 11 j E 480 1129 m 16 15 l 481 1128 m 10 11 j 509 1132 m 16 15 l 510 1132 m 11 11 j 539 1134 m 16 14 l 540 1132 m 11 11 j 570 1134 m 17 13 l 571 1132 m 10 11 j 599 1134 m 16 13 l 600 1132 m 11 11 j 629 1133 m 16 12 l 630 1131 m 11 12 j 659 1130 m 16 11 l 660 1128 m 12 12 j 690 1128 m 16 12 l 691 1126 m 11 14 j 720 1124 m 16 12 l 721 1122 m 11 14 j 750 1119 m 16 11 l 751 1117 m 11 15 j E 780 1112 m 16 10 l 781 1110 m 12 15 j 810 1106 m 15 11 l 811 1104 m 12 17 j 840 1097 m 15 10 l 841 1095 m 12 18 j 870 1087 m 15 10 l 871 1085 m 12 19 j 900 1076 m 15 10 l 901 1074 m 12 21 j 931 1062 m 16 9 l 932 1060 m 11 22 j 961 1047 m 16 9 l 962 1045 m 11 23 j 990 1030 m 15 8 l 991 1028 m 13 25 j 1020 1010 m 16 7 l 1021 1008 m 12 26 j 1049 989 m 15 7 l 1050 988 m 12 29 j E 237 1066 m 15 27 l 238 1066 m 9 9 j 265 1081 m 16 24 l 266 1081 m 10 9 j 294 1095 m 16 23 l 295 1095 m 9 9 j 322 1108 m 16 23 l 323 1108 m 9 10 j 351 1117 m 17 20 l 352 1117 m 10 9 j 380 1126 m 16 19 l 381 1126 m 10 11 j 409 1132 m 16 18 l 410 1132 m 10 10 j 438 1138 m 16 17 l 439 1138 m 11 10 j 468 1142 m 17 15 l 468 1142 m 11 11 j 497 1145 m 16 15 l 498 1145 m 10 11 j E 527 1146 m 17 13 l 528 1146 m 10 10 j 556 1147 m 16 12 l 557 1147 m 12 11 j 587 1147 m 16 12 l 588 1145 m 10 10 j 616 1147 m 16 12 l 617 1145 m 11 11 j 646 1145 m 16 11 l 647 1144 m 11 13 j 676 1143 m 16 12 l 677 1141 m 12 13 j 707 1140 m 16 12 l 708 1138 m 11 13 j 737 1135 m 16 10 l 738 1133 m 11 13 j 766 1130 m 15 10 l 767 1128 m 12 15 j 796 1124 m 15 11 l 797 1122 m 12 15 j E 827 1116 m 16 9 l 828 1114 m 11 16 j 857 1108 m 16 10 l 858 1106 m 11 18 j 887 1097 m 16 9 l 888 1095 m 12 19 j 917 1086 m 16 10 l 918 1084 m 12 21 j 947 1072 m 15 9 l 948 1070 m 12 22 j 977 1057 m 15 9 l 978 1055 m 11 24 j 1006 1039 m 15 8 l 1007 1037 m 13 26 j 1036 1020 m 15 9 l 1037 1018 m 12 28 j 254 1093 m 16 26 l 255 1093 m 9 10 j 282 1107 m 16 25 l 283 1107 m 10 10 j E 311 1119 m 16 23 l 312 1119 m 9 9 j 339 1129 m 16 20 l 340 1129 m 10 10 j 368 1138 m 16 20 l 369 1136 m 10 9 j 397 1145 m 16 18 l 398 1145 m 10 11 j 426 1150 m 16 17 l 427 1150 m 10 10 j 455 1154 m 16 15 l 456 1154 m 11 10 j 485 1158 m 17 15 l 485 1156 m 11 11 j 515 1160 m 17 15 l 516 1158 m 10 11 j 544 1161 m 16 14 l 545 1159 m 10 11 j 573 1161 m 16 13 l 574 1161 m 12 12 j E 604 1161 m 16 13 l 605 1159 m 10 11 j 633 1159 m 16 11 l 634 1159 m 11 12 j 663 1157 m 16 11 l 664 1155 m 11 11 j 693 1154 m 16 10 l 694 1152 m 12 11 j 724 1151 m 16 10 l 725 1149 m 11 13 j 753 1146 m 15 10 l 754 1145 m 11 14 j 783 1142 m 16 11 l 784 1140 m 12 15 j 813 1135 m 16 10 l 814 1133 m 12 16 j 844 1128 m 16 11 l 845 1126 m 11 17 j 874 1118 m 16 9 l 875 1116 m 11 18 j E 903 1108 m 15 10 l 904 1106 m 13 19 j 934 1096 m 16 9 l 934 1094 m 12 21 j 964 1082 m 16 9 l 965 1080 m 11 22 j 993 1066 m 15 8 l 994 1064 m 12 24 j 1022 1048 m 15 8 l 1023 1046 m 13 25 j 271 1117 m 16 24 l 272 1117 m 9 9 j 299 1129 m 16 21 l 300 1129 m 10 9 j 328 1141 m 16 21 l 329 1141 m 9 10 j 356 1149 m 16 19 l 357 1149 m 10 10 j 385 1156 m 16 17 l 386 1156 m 10 10 j E 414 1162 m 16 16 l 415 1162 m 10 11 j 443 1166 m 16 15 l 444 1166 m 10 10 j 472 1169 m 16 14 l 473 1169 m 11 10 j 502 1172 m 17 13 l 503 1172 m 11 10 j 532 1173 m 16 12 l 533 1173 m 10 10 j 561 1173 m 16 11 l 562 1173 m 10 10 j 590 1173 m 16 11 l 591 1173 m 12 11 j 621 1172 m 16 10 l 622 1170 m 10 10 j 650 1170 m 16 10 l 651 1170 m 11 11 j 680 1168 m 16 10 l 681 1166 m 12 11 j E 710 1165 m 16 10 l 711 1163 m 12 11 j 740 1162 m 15 10 l 741 1160 m 11 13 j 770 1157 m 16 10 l 771 1155 m 11 12 j 800 1152 m 16 9 l 801 1150 m 12 14 j 831 1145 m 17 9 l 831 1144 m 12 16 j 860 1138 m 15 10 l 861 1136 m 12 17 j 890 1128 m 15 9 l 891 1127 m 11 18 j 920 1118 m 16 9 l 921 1116 m 12 19 j 950 1106 m 16 9 l 951 1104 m 12 21 j 980 1092 m 15 9 l 981 1090 m 11 23 j E 1009 1076 m 15 9 l 1010 1074 m 12 25 j 288 1141 m 16 23 l 289 1141 m 9 10 j 316 1151 m 16 21 l 317 1151 m 10 9 j 345 1161 m 16 19 l 346 1161 m 9 10 j 373 1167 m 16 17 l 374 1167 m 10 10 j 402 1173 m 16 15 l 403 1173 m 10 10 j 432 1178 m 17 15 l 433 1178 m 9 10 j 461 1180 m 17 13 l 462 1180 m 9 9 j 490 1183 m 17 13 l 491 1183 m 11 10 j 520 1184 m 17 11 l 520 1184 m 11 10 j E 549 1185 m 16 11 l 550 1185 m 10 10 j 578 1185 m 16 11 l 579 1185 m 10 10 j 607 1184 m 16 10 l 608 1184 m 12 11 j 638 1182 m 16 9 l 639 1182 m 10 10 j 667 1180 m 16 9 l 668 1180 m 11 11 j 697 1179 m 16 10 l 698 1179 m 12 12 j 727 1176 m 16 10 l 728 1174 m 11 11 j 757 1172 m 16 9 l 758 1170 m 11 12 j 787 1167 m 16 9 l 788 1165 m 11 12 j 817 1162 m 16 9 l 818 1160 m 12 14 j E 847 1156 m 16 10 l 848 1154 m 11 15 j 877 1147 m 16 8 l 878 1145 m 11 16 j 907 1139 m 16 10 l 908 1137 m 11 18 j 936 1128 m 15 9 l 937 1127 m 12 20 j 967 1116 m 16 9 l 968 1114 m 11 21 j 996 1102 m 15 9 l 997 1100 m 11 24 j 305 1163 m 16 21 l 306 1163 m 9 10 j 333 1171 m 16 19 l 334 1171 m 10 9 j 362 1179 m 16 17 l 363 1179 m 9 10 j 390 1184 m 16 15 l 391 1184 m 10 10 j E 419 1189 m 16 15 l 420 1189 m 11 10 j 449 1192 m 16 13 l 450 1192 m 10 11 j 478 1194 m 16 13 l 479 1194 m 10 10 j 507 1196 m 16 12 l 508 1196 m 11 11 j 537 1196 m 17 11 l 537 1196 m 11 10 j 566 1196 m 16 10 l 567 1196 m 10 10 j 595 1195 m 16 9 l 596 1195 m 10 10 j 624 1194 m 16 9 l 625 1194 m 12 11 j 655 1193 m 16 10 l 656 1193 m 10 11 j 684 1191 m 16 10 l 685 1191 m 11 11 j E 714 1188 m 16 8 l 715 1188 m 12 11 j 744 1185 m 16 8 l 744 1185 m 12 11 j 774 1181 m 16 8 l 775 1180 m 11 12 j 803 1177 m 15 9 l 804 1175 m 12 12 j 833 1171 m 15 8 l 834 1169 m 12 12 j 864 1165 m 16 8 l 865 1163 m 11 15 j 893 1158 m 15 10 l 894 1156 m 12 16 j 923 1148 m 15 8 l 924 1146 m 11 17 j 952 1138 m 15 9 l 953 1136 m 13 19 j 983 1126 m 15 9 l 984 1124 m 11 21 j E 321 1182 m 15 19 l 322 1182 m 10 10 j 350 1189 m 16 17 l 351 1189 m 10 9 j 379 1195 m 16 15 l 380 1195 m 9 10 j 407 1199 m 16 14 l 408 1199 m 10 9 j 436 1202 m 16 12 l 437 1202 m 11 9 j 466 1204 m 16 11 l 467 1204 m 10 9 j 495 1205 m 16 10 l 496 1205 m 10 8 j 524 1206 m 16 9 l 525 1206 m 11 9 j 554 1206 m 17 9 l 554 1206 m 11 9 j 583 1205 m 16 8 l 584 1205 m 10 9 j E 612 1204 m 16 8 l 613 1204 m 10 9 j 641 1203 m 16 8 l 642 1203 m 12 9 j 672 1201 m 16 7 l 673 1201 m 10 9 j 701 1199 m 16 7 l 702 1199 m 11 10 j 730 1197 m 15 8 l 731 1197 m 12 11 j 761 1194 m 17 8 l 762 1194 m 11 12 j 790 1190 m 15 8 l 791 1190 m 11 11 j 820 1185 m 16 7 l 821 1183 m 11 11 j 849 1180 m 15 8 l 850 1179 m 13 13 j 880 1174 m 15 8 l 881 1172 m 11 13 j E 910 1167 m 16 8 l 911 1165 m 11 16 j 939 1158 m 15 9 l 940 1156 m 12 17 j 969 1147 m 16 8 l 969 1145 m 13 18 j 338 1199 m 16 15 l 339 1199 m 10 9 j 367 1205 m 16 15 l 368 1205 m 10 9 j 396 1210 m 16 14 l 397 1210 m 9 10 j 425 1213 m 17 13 l 426 1213 m 9 10 j 453 1215 m 16 12 l 454 1215 m 11 10 j 483 1215 m 16 10 l 484 1215 m 10 9 j 512 1215 m 16 9 l 513 1215 m 10 8 j E 541 1215 m 16 8 l 542 1215 m 11 8 j 571 1215 m 17 8 l 572 1215 m 10 9 j 600 1214 m 16 8 l 601 1214 m 10 9 j 629 1213 m 16 8 l 630 1213 m 11 9 j 658 1211 m 15 7 l 659 1211 m 12 9 j 688 1209 m 14 7 l 689 1209 m 11 9 j 718 1207 m 15 7 l 719 1207 m 10 9 j 747 1204 m 15 6 l 748 1204 m 12 9 j 777 1201 m 15 6 l 778 1201 m 11 10 j 807 1198 m 16 7 l 808 1198 m 11 11 j E 836 1194 m 15 8 l 837 1194 m 11 12 j 866 1189 m 16 8 l 867 1187 m 12 12 j 896 1183 m 15 8 l 897 1181 m 12 13 j 926 1176 m 15 8 l 927 1174 m 11 15 j 955 1167 m 15 8 l 956 1165 m 12 17 j 355 1215 m 16 15 l 356 1215 m 10 9 j 384 1219 m 16 13 l 385 1219 m 10 8 j 413 1222 m 16 11 l 414 1222 m 10 8 j 442 1223 m 16 9 l 443 1223 m 9 8 j 470 1224 m 16 9 l 471 1224 m 11 8 j E 500 1224 m 16 8 l 501 1224 m 10 8 j 529 1224 m 16 8 l 530 1224 m 10 8 j 558 1223 m 15 7 l 559 1223 m 11 8 j 588 1222 m 16 7 l 589 1222 m 10 7 j 617 1220 m 15 5 l 618 1220 m 10 6 j 646 1219 m 15 5 l 647 1219 m 11 7 j 675 1217 m 15 5 l 676 1217 m 11 7 j 705 1215 m 15 5 l 706 1215 m 11 7 j 734 1214 m 14 6 l 735 1214 m 11 9 j 763 1212 m 14 7 l 764 1212 m 12 10 j E 794 1209 m 15 7 l 795 1209 m 11 10 j 823 1206 m 14 7 l 824 1206 m 11 11 j 853 1202 m 16 7 l 854 1202 m 11 11 j 883 1198 m 16 8 l 883 1196 m 12 12 j 912 1192 m 15 8 l 913 1190 m 12 13 j 942 1184 m 15 7 l 943 1182 m 11 14 j 372 1229 m 16 13 l 373 1229 m 10 9 j 401 1231 m 16 11 l 402 1231 m 10 8 j 430 1232 m 16 9 l 431 1232 m 10 8 j 459 1233 m 16 9 l 460 1233 m 9 8 j E 487 1232 m 15 7 l 488 1232 m 11 7 j 517 1232 m 15 7 l 518 1232 m 10 7 j 546 1230 m 15 5 l 547 1230 m 10 6 j 574 1229 m 14 5 l 575 1229 m 12 6 j 604 1228 m 15 5 l 605 1228 m 11 7 j 633 1226 m 14 5 l 634 1226 m 11 6 j 662 1224 m 14 5 l 663 1224 m 12 6 j 692 1223 m 15 5 l 693 1223 m 11 7 j 721 1221 m 14 5 l 722 1221 m 11 6 j 751 1219 m 15 4 l 752 1219 m 10 6 j E 780 1217 m 15 4 l 781 1217 m 12 7 j 810 1215 m 14 5 l 811 1215 m 11 8 j 839 1213 m 14 6 l 840 1213 m 12 10 j 869 1209 m 14 6 l 870 1209 m 12 11 j 899 1205 m 16 7 l 900 1205 m 11 11 j 928 1199 m 15 7 l 929 1198 m 12 13 j 389 1239 m 16 9 l 390 1239 m 10 7 j 418 1240 m 16 8 l 419 1240 m 10 7 j 447 1240 m 16 7 l 448 1240 m 10 7 j 476 1239 m 15 6 l 477 1239 m 9 6 j E 504 1238 m 15 5 l 505 1238 m 11 5 j 534 1236 m 15 4 l 535 1236 m 10 5 j 562 1235 m 14 4 l 563 1235 m 10 5 j 591 1233 m 15 4 l 592 1233 m 10 4 j 621 1232 m 15 4 l 622 1232 m 9 5 j 650 1230 m 15 4 l 651 1230 m 9 5 j 679 1229 m 15 5 l 680 1229 m 10 5 j 709 1227 m 16 4 l 710 1227 m 9 5 j 738 1226 m 15 5 l 739 1226 m 11 6 j 767 1224 m 14 5 l 768 1224 m 11 6 j E 796 1223 m 14 5 l 797 1223 m 12 7 j 826 1221 m 14 5 l 827 1221 m 11 7 j 855 1218 m 14 4 l 856 1218 m 12 8 j 885 1215 m 14 5 l 886 1215 m 12 9 j 915 1212 m 15 6 l 916 1212 m 11 12 j 406 1247 m 15 7 l 407 1247 m 10 6 j 435 1247 m 15 6 l 436 1247 m 10 6 j 464 1246 m 15 5 l 465 1246 m 10 6 j 493 1244 m 15 5 l 494 1244 m 9 5 j 521 1242 m 15 4 l 522 1242 m 10 5 j E 551 1240 m 15 4 l 552 1240 m 8 4 j 579 1238 m 15 3 l 580 1238 m 9 4 j 608 1236 m 15 3 l 609 1236 m 10 3 j 638 1234 m 15 2 l 639 1234 m 9 3 j 666 1233 m 14 3 l 667 1233 m 10 3 j 695 1232 m 14 3 l 696 1232 m 11 4 j 725 1231 m 15 4 l 726 1231 m 10 4 j 754 1230 m 14 4 l 755 1230 m 10 5 j 783 1229 m 14 5 l 784 1229 m 11 5 j 813 1228 m 15 5 l 813 1228 m 12 6 j E 842 1226 m 14 5 l 843 1226 m 11 7 j 871 1224 m 14 5 l 872 1224 m 12 8 j 901 1222 m 14 6 l 902 1222 m 12 9 j 423 1252 m 15 4 l 424 1252 m 9 4 j 452 1251 m 15 4 l 453 1251 m 10 4 j 481 1249 m 15 3 l 482 1249 m 9 4 j 509 1247 m 14 3 l 510 1247 m 10 4 j 538 1244 m 15 2 l 539 1244 m 10 4 j 567 1242 m 14 2 l 568 1242 m 9 4 j 596 1239 m 15 1 l 597 1239 m 9 3 j E 624 1237 m 14 1 l 625 1237 m 11 3 j 654 1236 m 14 2 l 655 1236 m 9 3 j 683 1234 m 15 1 l 684 1234 m 9 2 j 711 1233 m 14 1 l 712 1233 m 11 2 j 741 1233 m 14 2 l 742 1233 m 10 3 j 770 1233 m 14 3 l 771 1233 m 10 4 j 799 1232 m 14 3 l 800 1232 m 11 4 j 829 1231 m 15 3 l 830 1231 m 10 4 j 858 1231 m 14 5 l 859 1231 m 11 6 j 887 1229 m 14 5 l 888 1229 m 12 6 j E 440 1256 m 15 4 l 441 1256 m 10 4 j 469 1253 m 15 2 l 470 1253 m 9 3 j 498 1250 m 15 1 l 499 1250 m 8 2 j 526 1247 m 15 w 527 1247 m 10 3 j 554 1244 m 14 w 555 1244 m 10 2 j 584 1242 m 15 w 585 1242 m 9 3 j 612 1239 m 14 w 613 1239 m 10 2 j 641 1237 m 15 w 642 1237 m 10 1 j 670 1236 m 14 w 671 1236 m 10 2 j 699 1235 m 14 1 l 700 1235 m 10 2 j E 728 1234 m 15 1 l 729 1234 m 10 1 j 757 1233 m 14 w 758 1233 m 10 x 786 1233 m 14 w 787 1233 m 10 1 j 815 1233 m 14 1 l 816 1233 m 11 2 j 845 1233 m 14 2 l 846 1233 m 10 2 j 874 1233 m 14 2 l 875 1233 m 10 3 j 457 1256 m 15 w 458 1256 m 10 3 j 485 1253 m 14 w 486 1253 m 10 3 j 515 1250 m 15 w 516 1250 m 7 2 j 543 1246 m 15 1 k 544 1246 m 6 1 j E 572 1243 m 16 1 k 572 1243 m 0 y 600 1240 m 14 2 k 601 1240 m 0 y 627 1238 m 13 1 k 656 1237 m 13 w 684 1235 m 6 w 676 1236 m 4 w 688 1235 m 9 x 715 1234 m 14 1 k 716 1234 m 11 x 744 1233 m 14 1 k 745 1233 m 10 x 773 1234 m 14 1 l 774 1234 m 10 1 j 802 1234 m 14 1 l 803 1234 m 10 1 j 831 1235 m 14 2 l 832 1235 m 11 2 j 861 1236 m 14 3 l E 862 1236 m 10 3 j 474 1254 m 15 2 k 475 1254 m 0 y 501 1251 m 10 1 k 489 1253 m 2 w 529 1248 m 9 1 k 519 1250 m 2 w 553 1245 m 2 w 547 1246 m 2 w 574 1243 m 1 w 603 1240 m 1 w 768 1234 m 3 x 787 1234 m 12 w 791 1234 m 9 x 818 1234 m 14 w 819 1234 m 11 1 h 847 1235 m 14 w 848 1235 m 11 1 h 477 1254 m 1 w E 1101 835 m 556 207 l 389 268 k 138 1206 m 18 310 j E 545 628 m 7 20 j 665 673 m 8 20 j 780 716 m 8 20 j 892 757 m 7 20 j 998 797 m 7 21 j 1101 835 m 7 20 j E 20.4278 Sr 20.4278 Sh ( 0) 567 568 0.5 T (0.2) 687 612 0.5 T (0.4) 803 656 0.5 T (0.6) 914 697 0.5 T (0.8) 1020 736 0.5 T (1) 1123 775 0.5 T (X1) 860 632 0.5 T 0 Sr 0 Sh 545 628 m 12 17 l 458 687 m 12 18 l 377 744 m 12 18 l 299 797 m 12 18 l 226 848 m 13 18 l 156 896 m 13 18 l E -34.4849 Sr -34.4849 Sh ( 0) 509 575 0.5 T (0.2) 422 634 0.5 T (0.4) 340 691 0.5 T (0.6) 263 744 0.5 T (0.8) 189 795 0.5 T (1) 119 843 0.5 T (X2) 290 674 0.5 T 0 Sr 0 Sh 156 896 m 22 1 l 153 946 m 22 1 l 150 996 m 21 1 l 146 1048 m 21 1 l 144 1100 m 21 1 l 140 1152 m 21 1 l 138 1206 m 22 1 l E 93.3153 Sr 93.3153 Sh (4) 92 892 0.5 T (6) 89 942 0.5 T (8) 86 992 0.5 T (10) 82 1044 0.5 T (12) 80 1096 0.5 T (14) 76 1148 0.5 T (16) 74 1202 0.5 T (Y) 40 1045 0.5 T 2 St 0 Sr 0 Sh 545 628 m 4 344 k 156 896 m 18 310 k 1101 835 m 20 319 h 681 1056 m 3 288 h E 545 628 m 389 268 k 541 972 m 403 234 k 1101 835 m 420 221 k 1121 1154 m 437 190 k E 545 628 m 556 207 h 541 972 m 580 182 h 156 896 m 525 160 h 138 1206 m 546 138 h E 1 St 0.123 0.937 0.290506 0.766456 So 0.5 1 0.333333 0.666667 Sg 0 Sd B 1370 1094 m 28 33 j 28 31 j 28 29 j 28 27 j 28 25 j 28 23 j 28 20 j 28 19 j 28 16 j 28 14 j 28 12 j 28 9 j 28 8 j 28 5 j 28 3 j 28 2 j 28 2 h 28 3 h 28 5 h 28 8 h 28 9 h 28 12 h 28 14 h 27 16 h 28 19 h 28 20 h 28 23 h 28 25 h 28 27 h 28 29 h 28 31 h 28 33 h E 1 Sd (X3) 1818 683 0.5 T 90 Sr 90 Sh (Y) 1209 956 0.5 T 0 Sr 0 Sh 1370 807 m 3 z 1549 807 m 3 z 1728 807 m 3 z 1907 807 m 3 z 2086 807 m 3 z 2265 807 m 3 z E 1370 807 m 895 x E (0.0) 1370 754 0.5 T (0.2) 1549 754 0.5 T (0.4) 1728 754 0.5 T (0.6) 1907 754 0.5 T (0.8) 2086 754 0.5 T (1.0) 2265 754 0.5 T 1334 818 m 3 w 1334 874 m 3 w 1334 929 m 3 w 1334 984 m 3 w 1334 1039 m 3 w 1334 1094 m 3 w E 1334 818 m 276 y E (0) 1298 818 0.5 T (1) 1298 874 0.5 T (2) 1298 929 0.5 T (3) 1298 984 0.5 T (4) 1298 1039 0.5 T (5) 1298 1094 0.5 T Z %%Trailer %%Pages: 1 %%DocumentNeededResources: font Helvetica W %%EOF SHAR_EOF fi if test -f 'fig2.eps' then echo shar: "will not over-write existing file 'fig2.eps'" else cat << \SHAR_EOF > 'fig2.eps' %!PS-Adobe-3.0 EPSF-3.0 %%Title: (S-PLUS Graphics) %%Creator: S-PLUS %%For: (& O'Connor) %%CreationDate: Tue May 6 15:44:49 1997 %%BoundingBox: 20 270 592 522 %%Pages: (atend) %%DocumentData: Clean7Bit %%DocumentNeededResources: (atend) %%EndComments %%BeginProlog 150 dict begin gsave /bd{bind def}def /PVer 3200 def % drawing commands /I{Coord SetPage 1 setlinecap 1 setlinejoin LineTypes{RastersPerPoint ScaleArray}forall /Helvetica findfont PointSize RastersPerPoint mul Cex mul scalefont setfont /Colors GColors def }bd /BWOp{ bw{currentrgbcolor add add 2.9 gt{1}{0}ifelse setgray}if exec bw{CurSc Sc}if }bd /IBu{}def /B{newpath}bd /E{{stroke}BWOp}bd /C{currentpoint Penup{newpath}{E}ifelse moveto}bd /F{closepath eofill}bd /M{moveto}bd /L{lineto}bd /Pu{load /XEQ exch def /Penup true def}bd /Pd{/XEQ{}def /Penup false def}bd /Px{currentdict /XEQ known{XEQ}if}bd /h{Penup{rmoveto Px}{rlineto}ifelse}bd /j{neg Penup{rmoveto Px}{rlineto}ifelse}bd /k{exch neg exch Penup{rmoveto Px}{rlineto}ifelse}bd /l{neg exch neg exch Penup{rmoveto Px}{rlineto}ifelse}bd /m{moveto Penup{Px}if}bd /x{0 Penup{rmoveto Px}{rlineto}ifelse}bd /y{0 exch Penup{rmoveto Px}{rlineto}ifelse}bd /w{neg x}bd /z{neg y}bd /a{B M dup y exch x neg y closepath}bd /b{a eofill}bd /c{a E}bd /A{PageBegin BackCol 0 ge {CurSc gsave 0 SC Page aload pop exch 4 -1 roll dup 5 1 roll sub 3 1 roll exch dup 4 1 roll sub 4 2 roll b grestore SC}if }bd /Ph{gsave Pch-x Pch-y rmoveto Show grestore}bd /Pc{Pch Ph}bd /T{/Adjust exch def gsave translate StringRot rotate 0 0 moveto dup stringwidth pop neg Adjust mul 0 rmoveto currentpoint translate TextShow grestore}bd /X{erasepage InPage{PageEnd}if}bd /Z{gsave showpage grestore PageEnd}bd /W{grestore end}bd % parameter setting commands /saveAt{Attr begin /Atlinewidth currentlinewidth def /Atfont currentfont def /Atdash currentdash 2 array astore def /AtSc CurSc def end}bd /restAt{Attr begin Atlinewidth Atfont Atdash AtSc end SC aload pop setdash setfont setlinewidth }bd /SC{dup dup 0 eq {pop BackIdx {BackCol dup 0 lt {pop 1 setgray} {Sc}ifelse} {/Colors BColor def 1 Sc}ifelse} {/Colors GColors def Sc}ifelse /CurSc exch def }bd /St{1 sub LineTypes dup 3 1 roll length Rem floor get 0 setdash}bd /Sw{abs 2 div RastersPerPoint mul setlinewidth SetClip}bd /Sp{dup 42 eq{pop IBu}{Pch exch 0 exch put SetPchSize}ifelse}bd /Sx{dup Cex div /Ratio exch def /Cex exch def currentfont Ratio scalefont setfont /Pch-x Pch-x Ratio mul def /Pch-y Pch-y Ratio mul def /Text-y Text-y Ratio mul def}bd /So{4 1 roll exch 4 -1 roll Plot astore pop SetClip}bd /Sg{4 1 roll exch 4 -1 roll Figure astore pop SetClip}bd /Sr{/StringRot exch def}bd /Sh{/CharRot exch def}bd /Sd{0 eq /ClipToPlot exch def SetClip}bd /Iv{/InPage false def/Clip 4 array def/Page 4 array def /Figure [0 0 1 1] def/Plot [0 0 1 1] def/ClipToPlot true def /Cex 1 def/Outline false def /Pch 1 string def/Pch-x 0 def/Pch-y 0 def/Text-y 0 def /LineTypes [ % in default units [] [1 2] [4 4] [8 4] [13 3] [16 2 2 2] [8 2 2 2] [1 13] [6 5] [12 4] ] def /Attr 5 dict def/CurSc 1 def/Penup false def/XEQ{}def }bd /Rem{2 copy div floor mul sub floor cvi}bd /RastersPerPoint{RastersPerInch 72 div}bd /ScaleArray{/Factor exch def /Array exch def 0 1 Array length 1 sub {dup Array exch get Factor mul Array 3 1 roll put}for}bd /Coord{Region aload pop /uy exch def /ux exch def /ly exch def /lx exch def uy ly sub ux lx sub Landscape{exch}if /Width exch def /Height exch def lx ly translate Landscape{90 rotate 0 Height neg translate}if 1 RastersPerPoint div dup scale}bd /SetPchSize{gsave newpath 0 0 moveto Pch false charpath flattenpath pathbbox exch 3 1 roll add 2 div neg /Pch-y exch def add 2 div neg /Pch-x exch def grestore}bd /Show{Outline{false charpath E}{{show}BWOp}ifelse}bd /SimpleShow{0 Text-y 2 div neg rmoveto Show}bd /FancyShow{ /RotDiff exch def /Cos RotDiff cos abs def /Sin RotDiff sin abs def { ( ) dup 0 4 -1 roll put dup stringwidth pop /CharWidth exch def Cos 0 eq{ Text-y Sin div }{ Sin 0 eq{ CharWidth Cos div }{ Text-y Sin div dup CharWidth Cos div dup 4 1 roll lt{exch pop}{pop}ifelse }ifelse }ifelse 2 div /CharDist exch def CharDist 0 translate 0 0 moveto gsave RotDiff rotate CharWidth 2 div neg Text-y 2 div neg rmoveto Show grestore CharDist 0 translate 0 0 moveto }forall }bd /TextShow{ CharRot StringRot sub dup 0 eq{pop SimpleShow}{FancyShow}ifelse }bd /BoxClip{/CLW currentlinewidth def 2{CLW add 4 1 roll}repeat 2{CLW sub 4 1 roll}repeat % initclip saveAt grestore gsave restAt newpath 2 index exch 2 index exch dup 6 index exch moveto 3{lineto}repeat closepath clip newpath}bd /Subregion{ 7 dict begin/A exch def/Uy exch def/Ux exch def/Ly exch def/Lx exch def Ux Lx sub A 0 get mul Lx add Uy Ly sub A 1 get mul Ly add Ux Lx sub A 2 get mul Lx add Uy Ly sub A 3 get mul Ly add end}bd /SetFigure{Page aload pop Figure Subregion}bd /SetPlot{SetFigure Plot Subregion}bd /SetClip{ClipToPlot{SetPlot}{SetFigure}ifelse BoxClip}bd /SetPage{0 0 Width Height Page astore RastersPerPoint ScaleArray}bd /PageBegin{save /PageContext exch def /InPage true def gsave}bd /PageEnd{grestore PageContext restore /InPage false def}bd /SImage{ dup /Bits exch def 8 exch idiv /PerByte exch def /Ht exch def /Wid exch def 1 Wid div 1 Ht div scale /saveSc CurSc def /Colors IColors def /curstr Wid PerByte add 1 sub PerByte idiv string def /MaskByte 16#ff Bits 8 sub bitshift def 1 1 Ht{ currentfile curstr readhexstring pop pop /CurPos 0 def /Left PerByte def 1 1 Wid{ Left PerByte eq{ /CurByte curstr CurPos get def /MaskMove -8 Bits add def /CurPos CurPos 1 add def }if CurByte MaskMove bitshift MaskByte and dup 0 gt{ Sc exch dup 3 1 roll B M 1 z 1 w 1 y F }{pop pop}ifelse Left 1 sub dup /Left exch def 0 le {/Left PerByte def} {/MaskMove MaskMove Bits add def}ifelse }for pop }for /Colors GColors def saveSc Sc }bd /Ic {} bd /Sc { dup dup 1 lt exch 0 ge and {1 exch sub setgray} % explicit gray level [0,1) {1 sub Colors dup 3 1 roll length Rem floor get dup type /arraytype eq {aload pop setrgbcolor} {setgray} ifelse } ifelse } bd /If {} bd /Sf {dup 0 lt /Outline exch def abs 1 sub Fonts dup 3 1 roll length Rem floor get findfont PointSize Cex mul RastersPerPoint mul scalefont dup setfont dup /FontMatrix get /Matrix exch def /FontBBox get aload pop Matrix transform 4 2 roll Matrix transform exch pop add /Text-y exch def pop SetPchSize} bd %%EndProlog %%BeginSetup % fixed controlling parameters /Landscape false def /Region [20.88 170.755 591.12 621.245] def /bw statusdict begin /processcolors where {pop processcolors} {1} ifelse end 1 eq def /RastersPerInch 300 def /PointSize 14 def /Fonts [ /Helvetica /Courier /Times-Roman /Helvetica-Oblique /Helvetica-Bold /Helvetica-BoldOblique /Courier-Oblique /Courier-Bold /Courier-BoldOblique /Times-Italic /Times-Bold /Times-BoldItalic /Symbol /AvantGarde-Book /AvantGarde-BookOblique /AvantGarde-Demi /AvantGarde-DemiOblique /Bookman-Demi /Bookman-DemiItalic /Bookman-Light /Bookman-LightItalic /Helvetica-Narrow /Helvetica-Narrow-Bold /Helvetica-Narrow-BoldOblique /Helvetica-Narrow-Oblique /NewCenturySchlbk-Roman /NewCenturySchlbk-Bold /NewCenturySchlbk-Italic /NewCenturySchlbk-BoldItalic /Palatino-Roman /Palatino-Bold /Palatino-Italic /Palatino-BoldItalic /ZapfChancery-MediumItalic /ZapfDingbats ] def /GColors [ 0 0.6 0.3 0.9 0.4 0.7 0.1 0.5 0.8 0.2 ] def /BackIdx true def /BackCol -1 def /IColors [ 0.933333 0.866667 0.8 0.733333 0.666667 0.6 0.533333 0.466667 0.4 0.333333 0.266667 0.2 0.133333 0.0666667 0 ] def /Colors GColors def /IBu {Pch 0 8#267 put SetPchSize} def /Bu {Pc} def % all initialization action here Iv I currentdict /Ic known {Ic} if currentdict /If known {If} if %%EndSetup %%Page: 1 1 %%PageOrientation: Portrait A 1 St 1 Sw 1 SC 0 Sr 42 Sp 1 Sx 0.123 0.937 0.0968354 0.922152 So 0 0.5 0 1 Sg 0 Sh 1 Sd %%IncludeResource: font Helvetica 1 Sf 545 645 m 13 15 k 532 661 m 14 15 k 518 677 m 14 15 k 13 16 k 491 709 m 13 14 k 478 724 m 13 14 k 465 739 m 14 14 k 451 754 m 13 14 k 438 769 m 13 14 k 425 784 m 13 13 k 412 798 m 13 15 k 399 814 m 13 13 k 386 828 m 12 14 k 374 843 m 12 13 k 362 857 m 14 13 k 348 871 m 12 13 k 336 885 m 12 14 k 323 899 m 11 10 k 311 909 m 11 8 k 299 917 m 11 8 k E 287 925 m 10 8 k 276 933 m 11 8 k 264 941 m 10 8 k 253 949 m 10 7 k 242 956 m 11 8 k 230 964 m 10 7 k 219 971 m 10 8 k 208 979 m 11 8 k 196 987 m 9 7 k 186 994 m 11 8 k 174 1002 m 9 6 k 164 1008 m 10 8 k 546 645 m 19 13 h 566 658 m 17 15 h 584 673 m 18 13 h 603 686 m 18 14 h 622 700 m 18 13 h 641 713 m 17 14 h 18 13 h 677 740 m 17 13 h E 695 753 m 17 14 h 713 767 m 17 13 h 731 780 m 18 13 h 750 793 m 17 13 h 768 806 m 16 13 h 785 819 m 17 13 h 803 832 m 17 14 h 821 846 m 17 12 h 839 858 m 16 13 h 856 871 m 17 13 h 874 884 m 16 13 h 891 897 m 16 12 h 908 909 m 17 12 h 926 921 m 16 13 h 943 934 m 16 12 h 960 946 m 15 1 h 976 947 m 15 1 h 992 948 m 15 1 h 1008 949 m 15 1 h 1024 950 m 15 1 h E 1040 951 m 15 1 h 1056 952 m 16 x 1073 952 m 14 1 h 1088 953 m 15 1 h 551 675 m 18 14 l 552 673 m 12 14 j 537 690 m 18 13 l 537 688 m 13 13 j 570 688 m 18 13 l 571 686 m 11 12 j 523 706 m 18 14 l 524 704 m 12 13 j 555 704 m 18 13 l 556 702 m 13 13 j 589 702 m 18 13 l 589 700 m 12 13 j 510 721 m 18 12 l 511 719 m 11 12 j 542 719 m 18 12 l 543 717 m 11 12 j E 574 717 m 18 12 l 575 715 m 13 12 j 606 715 m 17 12 l 607 713 m 13 12 j 497 736 m 18 12 l 498 734 m 11 12 j 528 734 m 17 12 l 529 732 m 12 12 j 561 732 m 18 12 l 562 730 m 11 12 j 593 730 m 18 12 l 594 728 m 12 12 j 625 728 m 18 12 l 626 727 m 13 13 j 483 751 m 17 12 l 484 749 m 12 12 j 515 749 m 17 12 l 516 747 m 11 12 j 547 747 m 18 12 l 548 745 m 12 12 j E 579 745 m 17 12 l 580 744 m 12 13 j 611 744 m 17 13 l 612 742 m 12 13 j 643 742 m 17 13 l 644 740 m 13 13 j 469 766 m 17 12 l 470 764 m 12 12 j 502 764 m 18 12 l 502 762 m 12 12 j 534 762 m 18 12 l 535 761 m 11 13 j 566 761 m 18 13 l 567 759 m 11 13 j 598 759 m 18 13 l 599 757 m 11 13 j 630 757 m 18 13 l 631 755 m 11 12 j 662 755 m 18 12 l 663 753 m 12 12 j E 457 781 m 18 12 l 458 779 m 10 12 j 488 779 m 18 12 l 489 778 m 12 13 j 520 778 m 18 13 l 521 776 m 12 13 j 552 776 m 17 13 l 553 774 m 12 12 j 584 774 m 17 12 l 585 772 m 12 12 j 616 772 m 17 12 l 617 770 m 12 12 j 648 770 m 17 12 l 649 768 m 12 12 j 680 768 m 17 12 l 681 766 m 12 12 j 444 796 m 18 12 l 445 794 m 11 12 j 475 795 m 17 13 l 476 793 m 11 13 j E 507 793 m 18 13 l 508 791 m 12 12 j 538 791 m 17 12 l 539 789 m 12 12 j 571 789 m 18 12 l 572 787 m 11 12 j 603 787 m 18 12 l 604 785 m 11 12 j 635 785 m 18 12 l 636 783 m 11 12 j 666 783 m 17 12 l 667 781 m 12 12 j 698 781 m 17 12 l 699 781 m 12 12 j 431 811 m 18 13 l 432 809 m 11 12 j 462 809 m 17 12 l 463 807 m 11 11 j 494 807 m 18 11 l 495 805 m 11 11 j E 525 805 m 17 11 l 526 803 m 11 11 j 557 804 m 18 12 l 558 802 m 12 12 j 589 802 m 17 12 l 589 800 m 13 12 j 621 800 m 17 12 l 622 798 m 12 12 j 653 798 m 17 12 l 654 797 m 11 13 j 685 797 m 18 13 l 686 795 m 11 13 j 717 795 m 18 13 l 718 793 m 11 12 j 417 825 m 17 11 l 418 823 m 12 11 j 450 824 m 18 12 l 451 822 m 10 12 j 481 822 m 18 12 l 482 820 m 11 12 j E 512 820 m 17 12 l 513 818 m 11 12 j 544 818 m 18 12 l 545 816 m 11 11 j 575 816 m 17 11 l 576 814 m 12 11 j 607 815 m 18 12 l 608 814 m 12 13 j 639 814 m 17 13 l 640 812 m 12 13 j 671 812 m 17 13 l 672 810 m 12 13 j 703 810 m 17 13 l 704 808 m 12 12 j 735 808 m 17 12 l 736 806 m 12 12 j 405 840 m 18 12 l 406 838 m 10 12 j 436 838 m 18 12 l 437 836 m 12 11 j E 468 836 m 17 11 l 468 834 m 12 11 j 499 834 m 17 11 l 500 832 m 11 11 j 531 833 m 18 12 l 532 831 m 11 12 j 562 831 m 17 12 l 563 830 m 11 13 j 593 830 m 17 13 l 594 830 m 12 13 j 625 828 m 17 12 l 626 826 m 12 12 j 657 826 m 17 12 l 658 826 m 12 12 j 689 825 m 17 12 l 690 823 m 12 12 j 721 823 m 17 12 l 722 821 m 12 12 j 753 821 m 17 12 l 754 819 m 12 12 j E 392 854 m 17 11 l 393 852 m 11 11 j 423 852 m 17 11 l 424 850 m 11 11 j 454 851 m 17 12 l 455 849 m 12 12 j 485 849 m 17 12 l 486 848 m 12 13 j 518 848 m 18 13 l 519 846 m 11 12 j 549 846 m 17 12 l 550 844 m 11 12 j 580 844 m 17 12 l 581 842 m 11 11 j 611 843 m 17 12 l 612 841 m 12 12 j 643 841 m 17 12 l 644 839 m 12 12 j 675 839 m 17 12 l 675 839 m 13 12 j E 707 837 m 17 11 l 708 837 m 12 12 j 739 835 m 17 11 l 740 835 m 12 12 j 770 834 m 16 12 l 771 832 m 12 12 j 380 868 m 17 11 l 381 866 m 10 11 j 411 867 m 18 12 l 412 866 m 10 13 j 442 866 m 18 13 l 443 864 m 10 12 j 473 864 m 18 12 l 474 862 m 11 12 j 504 862 m 18 12 l 505 860 m 12 11 j 536 861 m 17 12 l 537 859 m 11 12 j 567 859 m 17 12 l 568 857 m 11 12 j E 598 857 m 17 12 l 599 855 m 11 11 j 630 855 m 18 11 l 631 853 m 11 11 j 661 853 m 17 11 l 662 851 m 12 11 j 693 852 m 18 12 l 693 850 m 13 12 j 725 850 m 17 12 l 726 849 m 12 13 j 757 849 m 17 13 l 758 847 m 11 12 j 788 847 m 17 12 l 789 847 m 12 13 j 367 883 m 18 12 l 368 881 m 11 12 j 399 882 m 18 13 l 399 880 m 11 12 j 429 880 m 17 12 l 430 878 m 11 12 j E 460 878 m 17 12 l 461 876 m 11 11 j 491 876 m 17 11 l 492 874 m 11 11 j 522 875 m 17 12 l 523 873 m 12 11 j 554 873 m 17 11 l 554 871 m 12 11 j 585 871 m 17 11 l 586 869 m 11 11 j 616 869 m 17 11 l 617 869 m 12 12 j 648 868 m 17 12 l 649 866 m 11 12 j 679 866 m 17 12 l 680 865 m 12 12 j 710 865 m 17 12 l 711 865 m 13 13 j 743 863 m 17 12 l 744 863 m 12 13 j E 774 862 m 16 13 l 775 860 m 12 12 j 806 860 m 17 12 l 807 860 m 12 12 j 355 897 m 18 12 l 356 895 m 10 11 j 385 896 m 17 12 l 386 894 m 12 11 j 416 894 m 17 11 l 417 892 m 11 11 j 448 892 m 18 11 l 449 890 m 10 11 j 478 890 m 17 11 l 479 888 m 11 11 j 509 889 m 17 12 l 510 887 m 11 11 j 540 887 m 17 11 l 541 885 m 12 11 j 572 885 m 18 11 l 572 884 m 12 12 j E 603 884 m 17 12 l 604 883 m 11 13 j 634 883 m 17 13 l 635 883 m 12 13 j 665 881 m 16 12 l 666 881 m 12 13 j 697 879 m 17 12 l 698 877 m 12 11 j 728 878 m 17 12 l 729 876 m 13 12 j 760 876 m 16 12 l 761 876 m 12 12 j 792 874 m 17 11 l 793 874 m 12 12 j 824 872 m 17 11 l 825 872 m 12 12 j 343 911 m 18 11 l 344 909 m 10 11 j 373 909 m 17 11 l 374 907 m 10 10 j E 404 908 m 18 11 l 405 906 m 11 11 j 434 906 m 17 11 l 435 904 m 12 11 j 466 904 m 17 11 l 467 902 m 10 11 j 496 903 m 17 12 l 497 901 m 11 11 j 527 901 m 17 11 l 528 900 m 11 12 j 558 900 m 17 12 l 559 898 m 12 12 j 589 899 m 17 13 l 590 897 m 12 12 j 621 897 m 17 12 l 622 895 m 11 11 j 652 895 m 17 11 l 653 893 m 11 11 j 683 893 m 17 11 l 684 893 m 12 12 j E 714 892 m 16 12 l 715 892 m 12 12 j 746 890 m 17 11 l 747 888 m 12 11 j 778 888 m 17 11 l 779 888 m 12 12 j 810 887 m 17 12 l 811 885 m 12 12 j 841 885 m 16 12 l 842 885 m 12 12 j 330 921 m 18 11 l 331 921 m 11 9 j 361 923 m 17 11 l 362 921 m 10 11 j 391 922 m 17 12 l 392 920 m 11 11 j 422 920 m 17 11 l 423 919 m 10 12 j 452 919 m 17 12 l 453 917 m 12 12 j E 484 918 m 17 13 l 485 916 m 10 12 j 515 916 m 18 12 l 516 914 m 10 12 j 545 914 m 17 12 l 546 912 m 11 11 j 576 913 m 17 12 l 577 911 m 12 11 j 607 911 m 17 11 l 608 909 m 12 11 j 639 909 m 17 11 l 640 907 m 11 11 j 670 907 m 17 11 l 671 905 m 11 11 j 701 906 m 17 12 l 702 904 m 11 11 j 732 904 m 17 11 l 733 904 m 12 12 j 763 902 m 16 11 l 764 902 m 13 12 j E 796 901 m 17 12 l 796 901 m 13 12 j 827 900 m 16 12 l 828 900 m 12 13 j 859 898 m 17 12 l 860 898 m 12 13 j 318 929 m 17 11 l 319 929 m 11 7 j 348 934 m 17 12 l 349 934 m 11 10 j 379 936 m 17 12 l 380 934 m 10 11 j 410 935 m 18 12 l 411 933 m 10 12 j 440 933 m 17 12 l 441 931 m 10 12 j 470 931 m 17 12 l 471 931 m 12 12 j 502 930 m 17 11 l 502 928 m 12 11 j E 532 928 m 16 11 l 533 926 m 11 11 j 563 926 m 17 11 l 564 926 m 11 11 j 594 925 m 17 11 l 595 923 m 11 11 j 625 923 m 17 11 l 626 921 m 12 11 j 657 921 m 17 11 l 658 919 m 11 11 j 688 920 m 17 12 l 689 919 m 11 12 j 719 919 m 17 12 l 720 917 m 11 12 j 750 917 m 17 12 l 751 915 m 11 12 j 781 916 m 17 13 l 782 916 m 13 13 j 813 914 m 17 12 l 813 914 m 13 13 j E 844 912 m 16 11 l 845 912 m 13 13 j 876 911 m 16 12 l 877 911 m 12 12 j 307 937 m 18 11 l 308 937 m 9 7 j 336 941 m 17 11 l 337 941 m 10 6 j 366 946 m 17 11 l 367 946 m 11 10 j 397 948 m 17 12 l 398 948 m 11 12 j 428 947 m 17 11 l 429 945 m 10 11 j 458 945 m 17 11 l 459 943 m 10 11 j 488 943 m 17 11 l 489 943 m 12 11 j 520 942 m 18 11 l 520 940 m 11 11 j E 550 940 m 17 11 l 551 938 m 11 11 j 581 938 m 17 11 l 582 938 m 11 11 j 612 937 m 17 11 l 613 936 m 11 12 j 642 936 m 16 12 l 643 936 m 13 13 j 674 934 m 16 12 l 675 934 m 12 13 j 705 933 m 16 12 l 706 933 m 12 13 j 736 931 m 16 12 l 737 931 m 12 12 j 767 930 m 16 12 l 768 930 m 12 12 j 798 928 m 16 11 l 799 928 m 13 12 j 831 926 m 18 11 l 831 924 m 12 11 j E 862 925 m 17 12 l 863 925 m 12 12 j 893 923 m 16 11 l 894 923 m 12 12 j 295 945 m 17 11 l 295 945 m 11 7 j 325 950 m 17 12 l 326 950 m 9 8 j 355 954 m 18 12 l 356 954 m 9 7 j 384 958 m 17 11 l 385 958 m 11 9 j 415 960 m 17 11 l 416 960 m 11 11 j 446 959 m 17 11 l 447 957 m 10 11 j 476 957 m 17 11 l 477 955 m 10 11 j 506 955 m 17 11 l 507 955 m 12 11 j E 537 954 m 17 11 l 538 953 m 11 12 j 568 953 m 17 12 l 569 951 m 11 12 j 599 951 m 17 12 l 600 951 m 11 12 j 629 950 m 16 12 l 630 948 m 11 12 j 660 948 m 17 12 l 661 946 m 12 11 j 692 947 m 17 12 l 693 945 m 11 11 j 723 945 m 17 11 l 724 943 m 11 11 j 754 943 m 17 11 l 755 943 m 11 11 j 785 942 m 17 11 l 786 940 m 11 11 j 816 940 m 17 11 l 817 940 m 13 12 j E 848 938 m 17 11 l 848 938 m 13 12 j 879 937 m 16 11 l 880 937 m 12 12 j 910 936 m 16 12 l 911 936 m 13 14 j 283 953 m 17 11 l 284 953 m 10 7 j 312 957 m 17 11 l 313 957 m 11 6 j 343 961 m 17 10 l 344 961 m 10 7 j 373 966 m 17 12 l 374 966 m 9 7 j 402 971 m 17 12 l 403 971 m 11 10 j 433 972 m 17 11 l 433 972 m 12 11 j 464 971 m 17 11 l 465 969 m 10 11 j E 494 970 m 17 12 l 495 968 m 10 12 j 524 968 m 17 12 l 525 968 m 12 12 j 554 967 m 16 12 l 555 965 m 12 11 j 586 965 m 17 11 l 587 963 m 11 11 j 616 963 m 16 11 l 617 963 m 11 11 j 647 962 m 17 11 l 648 960 m 11 11 j 678 960 m 17 11 l 679 960 m 12 11 j 709 959 m 16 11 l 710 959 m 12 12 j 740 957 m 16 11 l 741 957 m 12 12 j 771 955 m 16 11 l 772 955 m 12 12 j E 802 954 m 16 11 l 803 954 m 12 12 j 833 953 m 16 12 l 834 953 m 13 13 j 865 951 m 17 12 l 865 951 m 13 13 j 896 950 m 16 12 l 897 950 m 12 13 j 927 948 m 16 12 l 928 948 m 13 13 j 272 960 m 17 10 l 273 960 m 9 6 j 301 965 m 17 11 l 302 965 m 10 7 j 330 970 m 17 12 l 331 970 m 11 8 j 361 973 m 17 11 l 362 973 m 10 6 j 391 978 m 17 11 l 392 978 m 9 7 j E 420 982 m 17 11 l 421 982 m 11 9 j 451 985 m 18 12 l 451 983 m 12 12 j 481 983 m 16 12 l 482 983 m 11 12 j 512 982 m 17 11 l 513 980 m 10 11 j 542 980 m 17 11 l 543 978 m 11 10 j 572 978 m 17 10 l 573 978 m 12 11 j 603 977 m 16 11 l 604 975 m 11 11 j 634 975 m 17 11 l 635 975 m 11 11 j 664 974 m 16 11 l 665 974 m 12 12 j 695 972 m 16 11 l 696 972 m 12 11 j E 727 971 m 17 11 l 727 970 m 12 12 j 757 970 m 16 12 l 758 970 m 12 13 j 788 968 m 16 12 l 789 968 m 12 13 j 819 967 m 16 12 l 820 967 m 12 13 j 850 965 m 16 11 l 851 965 m 13 12 j 882 964 m 17 12 l 883 964 m 12 12 j 913 962 m 16 11 l 914 962 m 12 12 j 944 960 m 16 11 l 945 960 m 13 13 j 261 969 m 18 12 l 261 969 m 10 8 j 290 972 m 17 11 l 291 972 m 9 6 j E 319 977 m 17 11 l 320 977 m 10 6 j 348 981 m 17 10 l 349 981 m 11 7 j 379 986 m 17 12 l 380 986 m 10 7 j 408 990 m 16 11 l 409 990 m 10 7 j 438 994 m 17 11 l 439 994 m 11 8 j 468 997 m 17 11 l 469 995 m 11 11 j 499 995 m 17 11 l 500 995 m 11 11 j 529 993 m 16 10 l 530 993 m 11 11 j 559 992 m 16 11 l 560 990 m 12 11 j 590 990 m 17 11 l 591 988 m 11 10 j E 621 989 m 17 11 l 622 988 m 11 12 j 651 988 m 16 12 l 652 988 m 11 12 j 682 987 m 17 12 l 683 985 m 11 12 j 712 985 m 16 12 l 713 985 m 13 13 j 744 983 m 17 11 l 744 983 m 12 12 j 775 982 m 17 11 l 776 980 m 11 11 j 805 980 m 16 11 l 806 980 m 12 12 j 836 979 m 16 11 l 837 979 m 12 12 j 867 977 m 16 11 l 868 977 m 13 12 j 899 976 m 16 11 l 900 976 m 12 12 j E 930 974 m 16 11 l 931 974 m 12 12 j 961 961 m 15 1 l 962 961 m 12 12 j 249 976 m 17 11 l 250 976 m 10 6 j 278 980 m 17 10 l 279 980 m 10 7 j 308 985 m 17 12 l 309 985 m 9 7 j 337 988 m 17 10 l 338 988 m 9 6 j 366 993 m 17 11 l 367 993 m 11 6 j 397 998 m 17 11 l 398 998 m 9 7 j 426 1002 m 17 11 l 427 1002 m 10 7 j 456 1006 m 17 11 l 457 1006 m 11 8 j E 486 1008 m 17 10 l 487 1006 m 11 10 j 517 1007 m 17 11 l 518 1006 m 10 12 j 547 1006 m 17 12 l 548 1004 m 10 11 j 577 1005 m 17 12 l 578 1003 m 11 12 j 607 1003 m 16 12 l 608 1003 m 12 12 j 638 1002 m 16 12 l 639 1000 m 11 12 j 669 1000 m 17 12 l 670 1000 m 11 12 j 699 998 m 16 10 l 700 998 m 11 11 j 730 997 m 17 11 l 731 995 m 12 11 j 761 995 m 17 11 l 762 995 m 12 12 j E 792 994 m 16 11 l 793 992 m 11 11 j 823 992 m 17 11 l 824 992 m 11 11 j 853 991 m 16 11 l 854 991 m 12 12 j 884 989 m 16 11 l 885 989 m 13 12 j 916 988 m 16 11 l 917 988 m 12 13 j 947 974 m 15 w 948 974 m 12 12 j 977 962 m 14 1 l 978 962 m 12 12 j 238 984 m 17 12 l 239 984 m 9 7 j 267 988 m 17 11 l 268 988 m 10 7 j 296 992 m 17 11 l 297 992 m 10 6 j E 326 996 m 17 10 l 327 996 m 9 7 j 355 1001 m 17 12 l 356 1001 m 9 7 j 384 1006 m 17 12 l 385 1006 m 11 7 j 414 1009 m 16 10 l 415 1009 m 10 6 j 444 1014 m 17 11 l 445 1014 m 10 7 j 474 1018 m 17 11 l 475 1018 m 10 9 j 503 1021 m 16 12 l 504 1021 m 12 12 j 534 1019 m 16 11 l 535 1019 m 11 12 j 564 1018 m 16 12 l 565 1018 m 11 12 j 594 1016 m 16 10 l 595 1016 m 11 11 j E 625 1015 m 17 11 l 626 1013 m 11 10 j 656 1013 m 17 10 l 657 1013 m 11 11 j 686 1012 m 16 11 l 687 1010 m 11 11 j 717 1010 m 17 11 l 718 1010 m 11 11 j 747 1009 m 16 11 l 748 1007 m 12 11 j 778 1007 m 16 11 l 779 1007 m 12 12 j 809 1006 m 16 11 l 810 1006 m 12 12 j 840 1005 m 16 12 l 841 1005 m 11 12 j 870 1003 m 16 11 l 871 1003 m 12 13 j 901 1002 m 16 12 l 902 1002 m 13 13 j E 933 988 m 16 w 934 988 m 12 12 j 963 975 m 14 1 l 964 975 m 12 12 j 993 963 m 14 1 l 994 963 m 12 12 j 226 990 m 17 10 l 227 990 m 10 5 j 256 995 m 17 10 l 257 995 m 9 7 j 284 1000 m 16 12 l 285 1000 m 10 7 j 313 1004 m 16 11 l 314 1004 m 11 7 j 343 1008 m 16 11 l 344 1008 m 10 6 j 372 1012 m 16 10 l 373 1012 m 10 6 j 402 1017 m 17 11 l 403 1017 m 10 7 j E 432 1022 m 17 12 l 433 1022 m 10 7 j 462 1025 m 17 10 l 463 1025 m 10 6 j 491 1030 m 16 11 l 492 1030 m 10 8 j 521 1032 m 17 10 l 522 1030 m 11 10 j 552 1031 m 17 11 l 553 1029 m 10 10 j 582 1029 m 17 10 l 583 1027 m 10 10 j 612 1028 m 17 11 l 613 1028 m 11 11 j 642 1026 m 16 10 l 643 1026 m 12 11 j 673 1025 m 16 11 l 674 1025 m 11 11 j 703 1023 m 16 10 l 704 1023 m 12 12 j E 734 1023 m 16 12 l 735 1023 m 11 12 j 764 1021 m 16 11 l 765 1021 m 12 12 j 796 1020 m 17 12 l 796 1020 m 12 12 j 826 1018 m 16 11 l 827 1018 m 12 12 j 857 1017 m 16 11 l 858 1015 m 11 11 j 887 1015 m 16 11 l 888 1015 m 12 12 j 918 1002 m 15 w 919 1002 m 13 13 j 949 989 m 15 1 l 950 989 m 12 12 j 979 976 m 14 1 l 980 976 m 12 12 j 1009 963 m 14 w 1010 963 m 12 12 j E 215 998 m 17 10 l 216 998 m 10 7 j 244 1003 m 17 12 l 245 1003 m 10 7 j 273 1006 m 16 10 l 274 1006 m 9 5 j 302 1011 m 17 10 l 303 1011 m 9 6 j 331 1016 m 17 11 l 332 1016 m 10 7 j 361 1020 m 17 11 l 362 1020 m 9 7 j 390 1024 m 17 11 l 391 1024 m 10 6 j 419 1028 m 16 10 l 420 1028 m 11 5 j 450 1033 m 17 10 l 451 1033 m 10 7 j 479 1038 m 16 12 l 480 1038 m 10 7 j E 509 1042 m 17 11 l 510 1042 m 10 9 j 538 1044 m 16 11 l 539 1044 m 12 11 j 569 1042 m 16 10 l 570 1042 m 11 11 j 599 1041 m 16 11 l 600 1041 m 11 11 j 629 1040 m 16 11 l 630 1040 m 11 12 j 659 1039 m 16 12 l 660 1039 m 12 12 j 690 1037 m 16 11 l 691 1037 m 11 12 j 721 1036 m 17 12 l 722 1036 m 11 12 j 751 1034 m 16 11 l 752 1034 m 11 11 j 781 1033 m 16 11 l 782 1033 m 13 12 j E 812 1031 m 16 10 l 813 1031 m 12 12 j 843 1030 m 16 11 l 844 1030 m 12 12 j 873 1028 m 15 10 l 874 1028 m 12 12 j 904 1016 m 15 1 l 905 1016 m 12 12 j 934 1003 m 14 1 l 935 1003 m 13 13 j 965 989 m 14 w 966 989 m 12 12 j 995 977 m 14 1 l 996 977 m 12 12 j 1025 964 m 14 1 l 1026 964 m 12 12 j 205 1006 m 17 11 l 206 1006 m 8 7 j 233 1010 m 17 11 l 234 1010 m 9 6 j E 261 1014 m 16 10 l 262 1014 m 10 7 j 291 1019 m 17 12 l 292 1019 m 9 7 j 320 1023 m 17 11 l 321 1023 m 9 6 j 349 1027 m 17 10 l 350 1027 m 10 6 j 379 1032 m 17 11 l 380 1032 m 9 7 j 408 1036 m 17 11 l 409 1036 m 9 7 j 437 1041 m 17 12 l 438 1041 m 11 7 j 467 1045 m 16 11 l 468 1045 m 10 6 j 497 1049 m 17 10 l 498 1049 m 10 6 j 526 1054 m 16 11 l 527 1054 m 10 9 j E 556 1056 m 17 11 l 557 1054 m 11 11 j 587 1055 m 17 12 l 588 1053 m 10 11 j 616 1053 m 16 11 l 617 1053 m 11 12 j 646 1052 m 16 11 l 647 1052 m 11 11 j 676 1050 m 16 10 l 677 1050 m 12 11 j 707 1049 m 16 11 l 708 1049 m 12 12 j 738 1048 m 16 11 l 739 1046 m 11 11 j 768 1046 m 16 11 l 769 1046 m 11 11 j 798 1045 m 16 11 l 799 1043 m 12 11 j 829 1043 m 16 11 l 830 1043 m 12 12 j E 860 1041 m 16 10 l 861 1041 m 11 11 j 890 1029 m 15 1 l 891 1029 m 12 12 j 920 1016 m 14 w 921 1016 m 13 12 j 951 1004 m 15 1 l 952 1004 m 12 13 j 981 990 m 14 1 l 982 990 m 12 12 j 1011 977 m 14 w 1012 977 m 12 12 j 1041 965 m 14 1 l 1042 965 m 13 12 j 193 1013 m 17 10 l 194 1013 m 10 7 j 222 1017 m 16 11 l 223 1017 m 9 6 j 251 1022 m 17 11 l 252 1022 m 9 7 j E 279 1025 m 17 10 l 280 1025 m 10 5 j 309 1030 m 17 10 l 310 1030 m 9 7 j 337 1035 m 16 12 l 338 1035 m 10 7 j 366 1039 m 16 11 l 367 1039 m 11 6 j 396 1043 m 16 10 l 397 1043 m 10 6 j 425 1048 m 16 11 l 426 1048 m 10 7 j 454 1052 m 16 11 l 455 1052 m 11 6 j 485 1057 m 17 11 l 485 1057 m 11 7 j 514 1061 m 16 11 l 515 1061 m 10 6 j 543 1065 m 16 10 l 544 1065 m 11 8 j E 573 1068 m 16 11 l 574 1068 m 12 11 j 604 1066 m 16 10 l 605 1064 m 10 10 j 634 1065 m 17 11 l 635 1063 m 10 10 j 664 1063 m 17 10 l 665 1061 m 10 10 j 693 1062 m 16 11 l 694 1062 m 12 11 j 724 1060 m 16 10 l 725 1060 m 12 11 j 755 1059 m 16 10 l 756 1059 m 11 11 j 785 1058 m 16 11 l 786 1058 m 11 11 j 815 1057 m 16 11 l 816 1057 m 12 12 j 846 1055 m 16 11 l 847 1055 m 12 13 j E 876 1042 m 14 1 l 877 1042 m 12 12 j 906 1029 m 14 w 907 1029 m 12 12 j 936 1017 m 14 1 l 937 1017 m 13 12 j 967 1004 m 15 w 968 1004 m 12 13 j 997 991 m 14 1 l 998 991 m 12 12 j 1027 978 m 14 1 l 1028 978 m 12 12 j 1056 966 m 13 1 l 1057 966 m 14 13 j 183 1020 m 17 11 l 184 1020 m 8 6 j 211 1024 m 17 10 l 212 1024 m 9 6 j 240 1029 m 17 11 l 241 1029 m 9 6 j E 268 1033 m 16 10 l 269 1033 m 9 7 j 297 1038 m 17 12 l 298 1038 m 10 7 j 326 1041 m 16 10 l 327 1041 m 9 5 j 355 1046 m 17 10 l 356 1046 m 9 6 j 384 1051 m 17 11 l 385 1051 m 10 7 j 414 1055 m 17 11 l 415 1055 m 9 6 j 443 1059 m 17 10 l 444 1059 m 9 6 j 472 1064 m 17 11 l 473 1064 m 11 6 j 502 1068 m 17 10 l 502 1068 m 11 6 j 531 1073 m 16 11 l 532 1073 m 10 7 j E 561 1077 m 17 11 l 562 1077 m 10 8 j 590 1079 m 16 10 l 591 1079 m 12 11 j 621 1078 m 16 11 l 622 1078 m 11 11 j 651 1076 m 16 10 l 652 1076 m 11 11 j 681 1076 m 16 12 l 682 1074 m 11 11 j 710 1074 m 16 11 l 711 1074 m 12 12 j 741 1073 m 16 12 l 742 1073 m 12 13 j 771 1071 m 15 11 l 772 1071 m 12 12 j 802 1070 m 16 11 l 803 1070 m 11 12 j 832 1068 m 16 10 l 833 1068 m 12 11 j E 863 1056 m 15 1 l 864 1056 m 11 12 j 892 1042 m 14 w 893 1042 m 12 12 j 922 1030 m 14 1 l 923 1030 m 12 12 j 952 1017 m 14 w 953 1017 m 13 12 j 983 1005 m 14 1 l 984 1005 m 12 13 j 1012 991 m 13 w 1013 991 m 13 12 j 1042 979 m 13 1 l 1043 979 m 12 12 j 1073 967 m 15 1 l 1073 967 m 13 13 j 172 1027 m 17 10 l 173 1027 m 9 6 j 200 1032 m 16 11 l 201 1032 m 9 7 j E 228 1036 m 16 11 l 229 1036 m 10 6 j 257 1041 m 16 11 l 258 1041 m 9 7 j 286 1044 m 17 10 l 287 1044 m 9 5 j 314 1049 m 16 10 l 315 1049 m 10 7 j 344 1053 m 17 11 l 345 1053 m 9 6 j 372 1058 m 16 11 l 373 1058 m 10 6 j 401 1062 m 16 10 l 402 1062 m 11 6 j 431 1066 m 16 10 l 432 1066 m 10 6 j 460 1071 m 16 11 l 461 1071 m 10 6 j 489 1076 m 16 11 l 490 1076 m 11 7 j E 519 1080 m 17 11 l 520 1080 m 10 6 j 549 1084 m 17 10 l 550 1084 m 10 6 j 578 1089 m 16 11 l 579 1089 m 10 9 j 607 1091 m 16 11 l 608 1091 m 12 12 j 638 1090 m 16 11 l 639 1090 m 11 12 j 668 1089 m 16 12 l 669 1089 m 11 12 j 698 1087 m 16 11 l 699 1085 m 11 10 j 727 1086 m 16 11 l 728 1086 m 12 11 j 758 1084 m 16 10 l 759 1084 m 11 11 j 788 1083 m 16 11 l 789 1083 m 12 12 j E 818 1081 m 15 10 l 819 1081 m 12 12 j 848 1069 m 14 1 l 849 1069 m 13 12 j 879 1056 m 14 w 880 1056 m 11 12 j 909 1043 m 15 1 l 910 1043 m 11 11 j 938 1030 m 14 w 939 1030 m 13 12 j 969 1018 m 15 1 l 969 1018 m 13 12 j 998 1006 m 13 1 l 999 1006 m 12 13 j 1028 992 m 14 1 l 1029 992 m 12 12 j 1058 980 m 14 1 l 1059 980 m 13 12 j 1088 967 m 14 w 1089 967 m 13 13 j E 190 1039 m 17 11 l 191 1039 m 8 6 j 218 1043 m 17 10 l 219 1043 m 8 6 j 246 1047 m 17 10 l 247 1047 m 9 6 j 275 1052 m 17 11 l 276 1052 m 9 7 j 303 1056 m 16 11 l 304 1056 m 9 6 j 332 1060 m 17 10 l 333 1060 m 10 6 j 361 1065 m 16 11 l 362 1065 m 9 7 j 390 1069 m 17 11 l 391 1069 m 9 6 j 418 1074 m 16 11 l 419 1074 m 11 7 j 448 1078 m 16 11 l 449 1078 m 10 6 j E 477 1082 m 16 10 l 478 1082 m 10 6 j 506 1087 m 16 11 l 507 1087 m 11 6 j 536 1092 m 16 11 l 537 1092 m 11 7 j 566 1095 m 16 10 l 567 1095 m 10 5 j 595 1100 m 16 10 l 596 1100 m 10 8 j 625 1103 m 17 11 l 626 1101 m 11 10 j 655 1101 m 16 10 l 656 1101 m 11 11 j 685 1100 m 16 10 l 686 1100 m 11 11 j 714 1098 m 15 10 l 715 1098 m 12 11 j 744 1097 m 16 10 l 745 1097 m 12 11 j E 775 1096 m 16 11 l 776 1096 m 11 11 j 805 1094 m 16 10 l 806 1094 m 11 11 j 835 1081 m 15 w 836 1081 m 12 11 j 865 1069 m 15 w 865 1069 m 13 12 j 895 1057 m 14 1 l 896 1057 m 12 13 j 925 1043 m 14 w 926 1043 m 11 11 j 954 1031 m 14 1 l 955 1031 m 13 12 j 985 1019 m 15 1 l 986 1019 m 11 12 j 1014 1006 m 14 w 1015 1006 m 12 13 j 1044 993 m 14 1 l 1045 993 m 12 12 j E 1073 981 m 13 1 l 1074 981 m 13 12 j 207 1050 m 16 10 l 208 1050 m 9 6 j 235 1055 m 16 11 l 236 1055 m 9 7 j 263 1058 m 16 10 l 264 1058 m 10 5 j 292 1063 m 16 10 l 293 1063 m 9 6 j 321 1067 m 17 10 l 322 1067 m 9 6 j 349 1072 m 16 11 l 350 1072 m 10 6 j 378 1076 m 16 10 l 379 1076 m 10 6 j 407 1080 m 16 10 l 408 1080 m 9 5 j 436 1085 m 17 10 l 437 1085 m 10 6 j E 465 1090 m 16 11 l 466 1090 m 10 7 j 494 1093 m 16 10 l 495 1093 m 10 5 j 523 1098 m 16 10 l 524 1098 m 11 5 j 553 1103 m 16 10 l 554 1103 m 11 7 j 583 1108 m 16 12 l 584 1108 m 10 7 j 612 1111 m 16 10 l 613 1111 m 11 7 j 642 1114 m 16 10 l 643 1112 m 11 10 j 672 1112 m 16 10 l 673 1112 m 11 11 j 702 1111 m 16 10 l 703 1110 m 10 11 j 731 1110 m 16 11 l 732 1110 m 12 11 j E 762 1109 m 17 11 l 762 1109 m 12 12 j 792 1108 m 16 11 l 793 1108 m 11 12 j 821 1094 m 14 w 822 1094 m 12 12 j 851 1082 m 14 1 l 852 1082 m 12 11 j 881 1069 m 15 w 882 1069 m 12 11 j 911 1057 m 14 w 912 1057 m 12 13 j 940 1044 m 13 1 l 941 1044 m 12 12 j 970 1031 m 14 w 971 1031 m 13 11 j 1000 1019 m 14 w 1001 1019 m 12 13 j 1030 1006 m 14 w 1031 1006 m 12 12 j E 1059 994 m 13 1 l 1060 994 m 13 12 j 225 1061 m 17 10 l 226 1061 m 8 5 j 253 1066 m 17 10 l 254 1066 m 8 7 j 281 1070 m 17 11 l 282 1070 m 9 6 j 310 1075 m 17 11 l 311 1075 m 9 7 j 338 1078 m 16 10 l 339 1078 m 9 5 j 366 1083 m 16 10 l 367 1083 m 10 6 j 396 1088 m 17 11 l 397 1088 m 9 7 j 424 1092 m 16 11 l 425 1092 m 10 6 j 453 1096 m 16 10 l 454 1096 m 10 5 j E 483 1101 m 17 10 l 484 1101 m 9 7 j 511 1105 m 16 11 l 512 1105 m 10 6 j 540 1110 m 16 11 l 541 1110 m 11 6 j 570 1114 m 16 10 l 571 1114 m 11 5 j 600 1119 m 16 10 l 601 1119 m 10 7 j 629 1123 m 16 11 l 630 1123 m 11 8 j 658 1126 m 15 11 l 659 1126 m 12 12 j 689 1125 m 16 12 l 690 1125 m 11 12 j 718 1123 m 15 11 l 719 1123 m 11 12 j 748 1122 m 16 11 l 749 1122 m 12 12 j E 779 1120 m 17 10 l 779 1120 m 12 11 j 808 1108 m 14 w 809 1108 m 11 12 j 838 1094 m 15 w 839 1094 m 11 11 j 867 1082 m 14 w 868 1082 m 12 11 j 897 1070 m 14 1 l 898 1070 m 12 12 j 927 1058 m 14 1 l 928 1058 m 11 12 j 956 1044 m 14 w 957 1044 m 12 12 j 986 1032 m 14 1 l 986 1032 m 13 12 j 1016 1020 m 14 1 l 1017 1020 m 12 13 j 1045 1006 m 13 w 1046 1006 m 12 11 j E 242 1073 m 16 11 l 243 1073 m 9 6 j 270 1077 m 16 10 l 271 1077 m 9 6 j 298 1081 m 16 10 l 299 1081 m 10 5 j 327 1086 m 16 10 l 328 1086 m 9 7 j 355 1090 m 16 11 l 356 1090 m 9 6 j 383 1094 m 16 10 l 384 1094 m 11 5 j 413 1099 m 16 10 l 414 1099 m 9 6 j 441 1103 m 16 10 l 442 1103 m 10 6 j 470 1108 m 16 11 l 471 1108 m 11 6 j 500 1112 m 16 10 l 501 1112 m 9 6 j E 528 1116 m 16 10 l 529 1116 m 10 6 j 557 1121 m 16 11 l 558 1121 m 11 6 j 587 1126 m 16 11 l 588 1126 m 11 6 j 617 1130 m 16 10 l 618 1130 m 10 6 j 646 1135 m 16 11 l 647 1135 m 11 8 j 675 1137 m 16 10 l 676 1137 m 12 11 j 706 1136 m 16 10 l 707 1134 m 10 10 j 735 1134 m 16 10 l 736 1134 m 11 11 j 765 1133 m 16 10 l 766 1133 m 12 11 j 795 1120 m 15 w 796 1120 m 11 11 j E 824 1108 m 14 w 825 1108 m 12 13 j 854 1095 m 14 1 l 855 1095 m 11 11 j 883 1083 m 14 1 l 884 1083 m 12 11 j 913 1070 m 14 w 914 1070 m 12 12 j 943 1058 m 14 w 944 1058 m 11 12 j 972 1045 m 14 1 l 973 1045 m 12 11 j 1002 1033 m 15 1 l 1003 1033 m 12 12 j 1031 1020 m 13 w 1032 1020 m 12 13 j 259 1084 m 16 10 l 260 1084 m 9 6 j 287 1088 m 16 10 l 288 1088 m 9 6 j E 315 1093 m 16 11 l 316 1093 m 10 6 j 344 1097 m 16 10 l 345 1097 m 9 6 j 372 1101 m 16 10 l 373 1101 m 9 6 j 400 1106 m 16 11 l 401 1106 m 11 6 j 430 1110 m 16 10 l 431 1110 m 9 6 j 458 1114 m 16 10 l 459 1114 m 10 5 j 487 1119 m 16 10 l 488 1119 m 11 6 j 517 1124 m 16 11 l 518 1124 m 9 7 j 545 1128 m 16 11 l 546 1128 m 10 6 j 574 1132 m 16 10 l 575 1132 m 11 5 j E 604 1137 m 16 10 l 605 1137 m 11 6 j 633 1142 m 15 11 l 634 1142 m 11 6 j 663 1146 m 16 10 l 664 1146 m 11 8 j 693 1148 m 17 10 l 693 1148 m 12 11 j 722 1147 m 15 10 l 723 1147 m 11 11 j 752 1145 m 16 10 l 753 1145 m 11 11 j 781 1133 m 14 w 782 1133 m 12 11 j 811 1121 m 15 1 l 812 1121 m 11 11 j 841 1108 m 15 w 842 1108 m 11 12 j 870 1095 m 14 w 871 1095 m 12 11 j E 900 1083 m 15 w 900 1083 m 12 11 j 929 1071 m 14 1 l 930 1071 m 12 13 j 958 1058 m 13 w 959 1058 m 12 12 j 987 1045 m 13 w 988 1045 m 13 11 j 1017 1033 m 13 w 1018 1033 m 12 12 j 276 1095 m 16 10 l 277 1095 m 9 6 j 304 1099 m 16 10 l 305 1099 m 9 6 j 332 1104 m 16 11 l 333 1104 m 10 6 j 361 1108 m 16 10 l 362 1108 m 9 6 j 389 1112 m 16 10 l 390 1112 m 9 5 j E 417 1117 m 16 10 l 418 1117 m 11 6 j 447 1121 m 16 10 l 448 1121 m 9 6 j 475 1126 m 16 11 l 476 1126 m 10 6 j 504 1130 m 16 10 l 505 1130 m 11 5 j 533 1135 m 15 10 l 534 1135 m 10 7 j 562 1139 m 16 11 l 563 1139 m 10 6 j 591 1144 m 16 11 l 592 1144 m 11 6 j 621 1148 m 16 10 l 622 1148 m 10 5 j 650 1153 m 16 10 l 651 1153 m 11 6 j 679 1158 m 15 11 l 680 1158 m 12 9 j E 710 1160 m 17 11 l 710 1160 m 11 11 j 739 1158 m 16 10 l 740 1158 m 11 12 j 768 1145 m 14 w 769 1145 m 11 11 j 797 1133 m 14 w 798 1133 m 12 11 j 827 1121 m 14 w 828 1121 m 12 12 j 856 1109 m 13 1 l 857 1109 m 12 13 j 886 1095 m 14 w 887 1095 m 12 11 j 916 1083 m 15 w 917 1083 m 11 11 j 945 1071 m 14 w 946 1071 m 11 12 j 974 1058 m 14 w 975 1058 m 11 11 j E 1004 1046 m 15 1 l 1004 1046 m 12 11 j 294 1106 m 17 10 l 295 1106 m 8 6 j 321 1110 m 16 10 l 322 1110 m 9 5 j 349 1115 m 16 10 l 350 1115 m 10 6 j 378 1119 m 16 10 l 379 1119 m 9 6 j 406 1124 m 16 11 l 407 1124 m 9 6 j 434 1128 m 16 10 l 435 1128 m 11 6 j 464 1132 m 16 10 l 465 1132 m 9 5 j 492 1137 m 16 10 l 493 1137 m 10 6 j 521 1142 m 16 11 l 522 1142 m 10 6 j E 550 1145 m 16 9 l 551 1145 m 10 5 j 579 1150 m 16 10 l 580 1150 m 10 5 j 608 1155 m 16 10 l 609 1155 m 11 6 j 638 1160 m 16 11 l 639 1160 m 10 6 j 667 1164 m 16 10 l 668 1164 m 10 5 j 696 1168 m 16 9 l 697 1168 m 12 7 j 726 1171 m 16 10 l 727 1171 m 11 11 j 755 1158 m 14 w 756 1158 m 11 12 j 784 1145 m 14 w 785 1145 m 11 11 j 813 1133 m 14 w 814 1133 m 12 11 j E 843 1121 m 14 w 844 1121 m 11 11 j 872 1109 m 14 w 873 1109 m 12 13 j 901 1096 m 13 1 l 902 1096 m 13 12 j 931 1084 m 14 1 l 932 1084 m 12 12 j 960 1071 m 13 w 961 1071 m 12 12 j 989 1058 m 13 w 990 1058 m 13 11 j 311 1117 m 16 10 l 312 1117 m 8 6 j 338 1122 m 16 11 l 339 1122 m 9 6 j 366 1126 m 16 10 l 367 1126 m 10 6 j 395 1130 m 16 10 l 396 1130 m 9 5 j E 423 1135 m 16 10 l 424 1135 m 9 6 j 451 1139 m 16 10 l 452 1139 m 11 6 j 481 1144 m 16 11 l 482 1144 m 9 6 j 509 1148 m 16 10 l 510 1148 m 10 5 j 537 1152 m 15 9 l 538 1152 m 11 6 j 567 1157 m 16 11 l 568 1157 m 10 6 j 596 1162 m 16 11 l 597 1162 m 10 6 j 624 1166 m 15 10 l 625 1166 m 12 5 j 654 1171 m 15 10 l 655 1171 m 11 6 j 683 1175 m 15 10 l 684 1175 m 11 6 j E 713 1180 m 16 11 l 714 1180 m 11 8 j 743 1171 m 16 w 744 1169 m 10 10 j 771 1158 m 14 w 772 1158 m 11 12 j 800 1145 m 14 w 801 1145 m 12 11 j 830 1133 m 15 w 831 1133 m 11 11 j 859 1121 m 14 w 860 1121 m 11 11 j 888 1109 m 14 w 889 1109 m 11 12 j 917 1096 m 14 w 918 1096 m 12 11 j 947 1084 m 14 w 948 1084 m 11 11 j 975 1072 m 13 1 l 976 1072 m 12 13 j E 327 1128 m 15 10 l 328 1128 m 9 5 j 355 1132 m 16 9 l 356 1132 m 9 5 j 383 1137 m 16 10 l 384 1137 m 10 6 j 412 1142 m 16 11 l 413 1142 m 9 6 j 440 1145 m 16 9 l 441 1145 m 10 5 j 468 1150 m 16 10 l 469 1150 m 11 5 j 497 1155 m 15 10 l 498 1155 m 10 6 j 526 1159 m 16 10 l 527 1159 m 10 6 j 554 1163 m 16 10 l 555 1163 m 11 5 j 584 1168 m 16 10 l 585 1168 m 10 5 j E 612 1173 m 15 10 l 613 1173 m 10 6 j 641 1178 m 16 11 l 642 1178 m 11 6 j 671 1181 m 16 9 l 672 1181 m 10 5 j 700 1186 m 16 10 l 701 1186 m 11 6 j 729 1180 m 14 w 730 1180 m 12 8 j 759 1171 m 15 w 760 1169 m 10 10 j 788 1159 m 15 1 l 789 1157 m 10 11 j 816 1145 m 14 w 817 1145 m 12 11 j 846 1133 m 15 w 847 1133 m 11 11 j 875 1121 m 14 w 876 1121 m 11 11 j E 904 1109 m 14 w 905 1109 m 12 12 j 933 1096 m 14 w 934 1096 m 12 11 j 962 1084 m 13 w 963 1084 m 11 11 j 344 1139 m 16 10 l 345 1139 m 9 6 j 372 1144 m 16 11 l 373 1144 m 9 6 j 400 1148 m 16 10 l 401 1148 m 10 5 j 429 1152 m 16 9 l 430 1152 m 9 6 j 457 1157 m 16 11 l 458 1157 m 10 6 j 485 1162 m 16 11 l 486 1162 m 10 6 j 514 1165 m 16 9 l 515 1165 m 10 5 j E 542 1170 m 15 10 l 543 1170 m 11 6 j 572 1175 m 17 11 l 572 1175 m 11 6 j 600 1180 m 15 11 l 601 1180 m 10 6 j 629 1183 m 16 9 l 630 1183 m 10 4 j 658 1188 m 16 9 l 659 1188 m 11 6 j 687 1193 m 15 11 l 688 1193 m 11 6 j 716 1186 m 14 w 717 1186 m 11 6 j 745 1180 m 14 w 746 1180 m 12 8 j 775 1171 m 15 w 776 1171 m 11 11 j 804 1159 m 15 w 805 1157 m 10 11 j E 832 1146 m 14 1 l 833 1146 m 12 11 j 862 1134 m 14 1 l 863 1134 m 11 11 j 891 1121 m 14 w 892 1121 m 11 11 j 919 1109 m 13 w 920 1109 m 12 12 j 949 1096 m 15 w 950 1096 m 11 11 j 361 1150 m 16 10 l 362 1150 m 9 5 j 389 1155 m 16 10 l 390 1155 m 9 6 j 417 1159 m 16 10 l 418 1159 m 10 6 j 445 1163 m 15 10 l 446 1163 m 10 5 j 473 1168 m 15 10 l 474 1168 m 11 5 j E 502 1172 m 16 9 l 502 1172 m 11 6 j 531 1177 m 16 11 l 532 1177 m 9 6 j 559 1181 m 16 10 l 560 1181 m 11 5 j 588 1186 m 16 10 l 589 1186 m 10 6 j 617 1190 m 16 10 l 618 1190 m 10 6 j 645 1195 m 15 11 l 646 1195 m 12 6 j 675 1199 m 16 10 l 675 1199 m 11 5 j 704 1193 m 15 w 705 1193 m 10 6 j 732 1186 m 14 w 733 1186 m 11 6 j 762 1180 m 15 w 762 1180 m 12 8 j E 791 1171 m 14 w 792 1171 m 11 11 j 819 1159 m 14 w 820 1159 m 11 12 j 848 1146 m 14 w 849 1146 m 12 11 j 877 1134 m 13 w 878 1134 m 12 12 j 906 1122 m 13 1 l 907 1122 m 11 12 j 935 1110 m 14 1 l 936 1110 m 12 12 j 378 1161 m 16 10 l 379 1161 m 9 5 j 405 1165 m 15 9 l 406 1165 m 10 5 j 433 1170 m 15 10 l 434 1170 m 10 6 j 462 1174 m 16 10 l 463 1174 m 9 5 j E 490 1179 m 16 10 l 491 1179 m 10 6 j 519 1183 m 17 10 l 520 1183 m 10 5 j 547 1188 m 15 10 l 548 1188 m 10 6 j 575 1192 m 15 10 l 576 1192 m 10 5 j 605 1197 m 16 10 l 606 1197 m 10 6 j 633 1201 m 15 10 l 634 1201 m 10 5 j 662 1205 m 16 9 l 663 1205 m 10 5 j 691 1199 m 15 w 692 1199 m 11 5 j 720 1193 m 14 w 721 1193 m 10 6 j 748 1186 m 14 w 749 1186 m 12 6 j E 778 1180 m 15 w 779 1180 m 11 8 j 807 1171 m 14 w 808 1171 m 10 10 j 835 1159 m 14 w 836 1159 m 12 12 j 865 1146 m 15 w 865 1146 m 11 10 j 893 1134 m 14 w 894 1134 m 11 11 j 921 1122 m 13 w 922 1122 m 12 12 j 395 1172 m 16 10 l 396 1172 m 8 6 j 422 1176 m 16 10 l 423 1176 m 10 5 j 451 1180 m 17 9 l 451 1180 m 10 5 j 479 1185 m 16 10 l 480 1185 m 9 5 j E 506 1190 m 15 10 l 507 1190 m 11 6 j 535 1194 m 15 10 l 536 1194 m 9 5 j 564 1198 m 16 9 l 565 1198 m 9 5 j 592 1203 m 16 10 l 593 1203 m 11 5 j 621 1208 m 15 10 l 622 1208 m 10 6 j 649 1212 m 15 10 l 650 1212 m 11 6 j 678 1205 m 14 w 679 1205 m 10 5 j 707 1198 m 14 1 k 708 1198 m 10 4 j 736 1193 m 14 w 737 1193 m 10 6 j 764 1186 m 14 w 765 1186 m 12 6 j E 794 1180 m 15 w 795 1180 m 11 8 j 822 1171 m 13 w 823 1171 m 11 11 j 851 1159 m 14 w 852 1159 m 12 12 j 880 1146 m 14 w 881 1146 m 11 11 j 908 1134 m 13 w 909 1134 m 11 11 j 411 1182 m 15 9 l 412 1182 m 9 5 j 439 1187 m 16 10 l 440 1187 m 10 6 j 467 1192 m 16 11 l 468 1192 m 10 6 j 495 1196 m 15 10 l 496 1196 m 9 5 j 523 1200 m 16 9 l 524 1200 m 10 5 j E 552 1205 m 16 10 l 553 1205 m 10 6 j 580 1210 m 15 11 l 581 1210 m 10 6 j 608 1214 m 15 10 l 609 1214 m 10 5 j 637 1218 m 15 9 l 638 1218 m 10 5 j 666 1212 m 15 w 667 1212 m 10 6 j 694 1205 m 14 w 695 1205 m 9 5 j 706 1199 m 0 y 723 1198 m 14 w 724 1198 m 10 4 j 752 1192 m 14 1 k 753 1192 m 9 5 j 780 1186 m 14 w 781 1186 m 12 6 j 810 1180 m 14 w E 811 1180 m 10 8 j 838 1171 m 14 w 839 1171 m 11 11 j 866 1159 m 13 w 867 1159 m 12 12 j 895 1146 m 13 w 896 1146 m 11 11 j 428 1194 m 16 11 l 429 1194 m 9 6 j 455 1198 m 15 10 l 456 1198 m 10 5 j 484 1202 m 16 9 l 485 1202 m 9 5 j 511 1207 m 15 10 l 512 1207 m 10 6 j 539 1211 m 15 10 l 540 1211 m 10 5 j 568 1215 m 15 9 l 569 1215 m 9 4 j 596 1220 m 15 9 l E 597 1220 m 10 5 j 624 1224 m 15 9 l 625 1224 m 10 5 j 654 1218 m 15 w 655 1218 m 10 5 j 682 1212 m 14 w 683 1212 m 10 6 j 710 1205 m 14 w 711 1205 m 11 6 j 739 1198 m 14 w 740 1198 m 9 4 j 751 1193 m 0 y 767 1192 m 13 w 768 1192 m 11 5 j 796 1186 m 14 w 797 1186 m 12 6 j 825 1180 m 13 w 826 1180 m 11 8 j 853 1171 m 13 w 854 1171 m 11 11 j E 882 1159 m 14 w 883 1159 m 11 12 j 444 1204 m 15 9 l 445 1204 m 9 6 j 472 1209 m 16 11 l 473 1209 m 10 6 j 500 1213 m 15 10 l 501 1213 m 9 5 j 528 1217 m 16 9 l 529 1217 m 9 5 j 556 1222 m 16 10 l 557 1222 m 10 6 j 584 1227 m 15 11 l 585 1227 m 10 6 j 612 1231 m 15 10 l 613 1231 m 10 6 j 641 1224 m 15 w 642 1224 m 10 5 j 670 1218 m 14 w 671 1218 m 10 5 j E 698 1212 m 14 w 699 1212 m 11 6 j 727 1205 m 15 w 727 1205 m 11 6 j 755 1198 m 14 w 756 1198 m 10 5 j 783 1192 m 14 w 784 1192 m 11 5 j 812 1186 m 14 w 813 1186 m 11 6 j 841 1180 m 14 w 842 1180 m 10 8 j 869 1171 m 14 w 870 1171 m 11 10 j 460 1215 m 15 10 l 461 1215 m 9 5 j 488 1219 m 15 9 l 489 1219 m 10 5 j 516 1224 m 15 10 l 517 1224 m 10 6 j E 544 1228 m 15 10 l 545 1228 m 9 5 j 572 1233 m 15 10 l 573 1233 m 10 5 j 601 1237 m 16 9 l 602 1237 m 9 5 j 629 1231 m 15 w 630 1231 m 11 6 j 658 1224 m 15 w 658 1224 m 10 5 j 686 1217 m 14 1 k 687 1217 m 9 4 j 714 1212 m 14 w 715 1212 m 11 6 j 743 1205 m 15 w 744 1205 m 10 6 j 771 1198 m 14 w 772 1198 m 10 5 j 799 1192 m 14 w 800 1192 m 10 5 j E 828 1185 m 15 1 k 829 1185 m 10 5 j 856 1180 m 13 w 857 1180 m 11 8 j 477 1225 m 16 10 l 478 1225 m 9 5 j 504 1230 m 15 10 l 505 1230 m 10 5 j 533 1234 m 16 9 l 534 1234 m 9 5 j 560 1239 m 15 10 l 561 1239 m 11 6 j 589 1243 m 16 10 l 589 1243 m 10 5 j 617 1236 m 14 1 k 618 1236 m 9 4 j 645 1231 m 14 w 646 1231 m 11 6 j 673 1224 m 14 w 674 1224 m 9 5 j E 685 1218 m 0 y 701 1217 m 13 w 702 1217 m 10 4 j 729 1211 m 13 1 k 730 1211 m 11 5 j 758 1205 m 14 w 759 1205 m 11 6 j 786 1198 m 13 w 787 1198 m 11 5 j 814 1192 m 13 w 815 1192 m 10 5 j 827 1186 m 0 y 843 1185 m 13 w 844 1185 m 10 5 j 493 1236 m 15 10 l 494 1236 m 9 5 j 520 1240 m 15 9 l 521 1240 m 10 5 j 549 1245 m 15 10 l 550 1245 m 9 5 j E 576 1250 m 15 10 l 577 1250 m 11 6 j 605 1243 m 15 w 606 1243 m 8 5 j 616 1237 m 0 y 633 1236 m 14 w 634 1236 m 9 4 j 661 1230 m 14 1 k 662 1230 m 10 5 j 689 1224 m 14 w 690 1224 m 10 6 j 717 1217 m 14 w 718 1217 m 9 4 j 728 1212 m 0 y 745 1211 m 14 w 746 1211 m 10 5 j 774 1205 m 14 w 775 1205 m 10 6 j 802 1198 m 14 w 803 1198 m 10 5 j E 830 1192 m 14 w 831 1192 m 11 6 j 509 1247 m 15 10 l 510 1247 m 10 6 j 537 1250 m 16 9 l 537 1250 m 10 4 j 565 1255 m 15 9 l 566 1255 m 9 5 j 592 1249 m 14 1 k 593 1249 m 10 5 j 621 1243 m 15 w 622 1243 m 10 6 j 649 1236 m 14 w 650 1236 m 8 4 j 660 1231 m 0 y 676 1230 m 13 w 677 1230 m 10 5 j 705 1224 m 14 w 706 1224 m 10 6 j 733 1217 m 14 w E 734 1217 m 10 5 j 761 1211 m 14 w 762 1211 m 10 5 j 789 1204 m 13 1 k 790 1204 m 10 5 j 817 1198 m 13 w 818 1198 m 11 5 j 525 1257 m 15 9 l 526 1257 m 10 6 j 553 1261 m 16 10 l 554 1261 m 9 5 j 581 1255 m 14 w 582 1255 m 7 5 j 591 1250 m 0 y 608 1249 m 14 w 609 1249 m 10 5 j 637 1242 m 14 1 k 638 1242 m 9 5 j 664 1236 m 13 w 665 1236 m 10 5 j E 693 1230 m 15 w 693 1230 m 10 5 j 721 1223 m 14 1 k 722 1223 m 9 5 j 748 1217 m 13 w 749 1217 m 11 5 j 777 1211 m 15 w 778 1211 m 8 5 j 788 1205 m 0 y 804 1204 m 13 w 805 1204 m 10 5 j 541 1267 m 15 9 l 542 1267 m 10 5 j 569 1261 m 15 w 570 1261 m 9 5 j 597 1254 m 14 1 k 598 1254 m 9 4 j 624 1249 m 14 w 625 1249 m 9 5 j 636 1243 m 0 y E 653 1242 m 14 w 654 1242 m 9 5 j 680 1235 m 14 1 k 681 1235 m 10 4 j 709 1230 m 15 w 710 1230 m 8 5 j 720 1224 m 0 y 736 1223 m 13 w 737 1223 m 9 5 j 764 1216 m 14 1 k 765 1216 m 10 4 j 792 1211 m 13 w 793 1211 m 10 6 j 557 1267 m 14 w 558 1267 m 10 5 j 585 1261 m 14 w 586 1261 m 8 5 j 596 1255 m 0 y 613 1254 m 14 w 614 1254 m 9 4 j E 641 1248 m 15 1 k 641 1248 m 10 5 j 668 1242 m 13 w 669 1242 m 8 5 j 679 1236 m 0 y 696 1235 m 14 w 697 1235 m 10 4 j 724 1230 m 14 w 725 1230 m 10 6 j 751 1223 m 13 w 752 1223 m 10 5 j 763 1217 m 0 y 779 1216 m 13 w 780 1216 m 10 4 j 573 1267 m 14 w 574 1267 m 10 5 j 601 1260 m 14 1 k 602 1260 m 9 5 j 628 1254 m 13 w 629 1254 m 9 4 j E 640 1249 m 0 y 657 1248 m 15 w 658 1248 m 9 5 j 684 1242 m 14 w 685 1242 m 10 6 j 711 1235 m 13 w 712 1235 m 10 4 j 739 1229 m 13 1 k 740 1229 m 9 5 j 767 1223 m 14 w 768 1223 m 11 6 j 589 1266 m 14 1 k 590 1266 m 8 4 j 600 1261 m 0 y 617 1260 m 14 w 618 1260 m 9 5 j 644 1253 m 14 1 k 645 1253 m 10 4 j 672 1248 m 14 w 673 1248 m 9 5 j E 699 1241 m 13 1 k 700 1241 m 10 5 j 727 1235 m 14 w 727 1235 m 9 4 j 738 1230 m 0 y 755 1229 m 14 w 756 1229 m 9 5 j 605 1266 m 14 w 606 1266 m 9 5 j 632 1260 m 13 w 633 1260 m 8 5 j 643 1254 m 0 y 659 1253 m 13 w 660 1253 m 10 4 j 687 1248 m 13 w 688 1248 m 8 5 j 698 1242 m 0 y 715 1241 m 14 w 716 1241 m 10 5 j 743 1234 m 15 1 k E 744 1234 m 9 4 j 621 1266 m 15 w 622 1266 m 9 5 j 648 1259 m 14 1 k 649 1259 m 9 5 j 675 1253 m 14 w 676 1253 m 10 4 j 703 1247 m 14 1 k 704 1247 m 9 5 j 730 1241 m 13 w 731 1241 m 9 5 j 742 1235 m 0 y 636 1265 m 13 1 k 637 1265 m 7 4 j 663 1259 m 13 w 664 1259 m 10 5 j 691 1252 m 14 1 k 692 1252 m 7 3 j 718 1247 m 13 w 719 1247 m 9 5 j E 652 1265 m 14 w 653 1265 m 9 5 j 679 1259 m 14 w 680 1259 m 8 5 j 690 1253 m 0 y 706 1252 m 13 w 707 1252 m 9 4 j 667 1265 m 13 w 668 1265 m 9 5 j 694 1258 m 13 1 k 695 1258 m 10 5 j 682 1264 m 13 1 k 683 1264 m 8 4 j E 1093 807 m 547 204 l 383 263 k 145 1172 m 18 306 j E 663 647 m 8 20 j 780 691 m 8 20 j 892 732 m 7 20 j 1000 772 m 7 20 j E 20.4584 Sr 20.4584 Sh (0.2) 686 587 0.5 T (0.4) 803 630 0.5 T (0.6) 914 672 0.5 T (0.8) 1022 712 0.5 T (Predictor 1) 857 605 0.5 T 0 Sr 0 Sh 468 656 m 12 18 l 386 713 m 12 17 l 308 767 m 12 17 l 233 818 m 12 17 l E -34.5625 Sr -34.5625 Sh (0.2) 432 603 0.5 T (0.4) 350 660 0.5 T (0.6) 271 714 0.5 T (0.8) 197 765 0.5 T (Predictor 2) 294 647 0.5 T 0 Sr 0 Sh 163 866 m 21 1 l 158 940 m 21 1 l 154 1016 m 22 1 l 149 1093 m 21 2 l 145 1172 m 21 1 l E 93.3665 Sr 93.3665 Sh ( 0) 99 863 0.5 T (5) 94 936 0.5 T (10) 90 1012 0.5 T (15) 85 1089 0.5 T (20) 81 1168 0.5 T (Response) 47 1013 0.5 T 2 St 0 Sr 0 Sh 546 603 m 4 339 k 163 866 m 18 306 k 1093 807 m 20 313 h 680 1023 m 3 285 h E 546 603 m 383 263 k 542 942 m 397 230 k 1093 807 m 413 216 k 1113 1120 m 430 188 k E 546 603 m 547 204 h 542 942 m 571 178 h 163 866 m 517 157 h 145 1172 m 538 136 h E 1 St 0.123 0.937 0.290506 0.766456 So 0.5 1 0.333333 0.666667 Sg 0 Sd B 1370 1094 m 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 10 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 10 6 j 9 7 j 9 6 j 9 6 j 9 6 j 9 7 j 9 6 j 9 6 j 9 7 j 9 4 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 10 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 10 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 4 h 9 5 h 9 5 h 9 5 h 9 5 h E 1 Sd (Predictor 3) 1818 683 0.5 T 90 Sr 90 Sh (Response) 1209 956 0.5 T 0 Sr 0 Sh 1357 807 m 3 z 1539 807 m 3 z 1722 807 m 3 z 1905 807 m 3 z 2088 807 m 3 z 2271 807 m 3 z E 1357 807 m 914 x E (0.0) 1357 754 0.5 T (0.2) 1539 754 0.5 T (0.4) 1722 754 0.5 T (0.6) 1905 754 0.5 T (0.8) 2088 754 0.5 T (1.0) 2271 754 0.5 T 90 Sr 90 Sh 1334 844 m 3 w 1334 907 m 3 w 1334 969 m 3 w 1334 1031 m 3 w 1334 1093 m 3 w E 1334 844 m 249 y E (17) 1281 844 0.5 T (19) 1281 969 0.5 T (21) 1281 1093 0.5 T 0 Sr 0 Sh 0 Sd 967 298 1334 807 c Z %%Trailer %%Pages: 1 %%DocumentNeededResources: font Helvetica W %%EOF SHAR_EOF fi if test -f 'is.polymars' then echo shar: "will not over-write existing file 'is.polymars'" else cat << \SHAR_EOF > 'is.polymars' .BG .FN is.polymars .TL test for polymars objects .CS is.polymars(object) .RA .AG object the object under investigation. .RT returns TRUE if the object is a polymars model, FALSE otherwise. .SA plot.polymars, predict.polymars, summary.polymars, print.polymars .WR SHAR_EOF fi if test -f 'library.help' then echo shar: "will not over-write existing file 'library.help'" else cat << \SHAR_EOF > 'library.help' This section contains programs for PolyMARS, (polychotomous) regression based on ``Multivariate Adaptive Regression Splines'' J.H. Friedman (1991) and ``Polychotomous Regression'' Kooperberg, C, Bose, S. and Stone C.J.. The functions were developed by Charles Kooperberg and Martin O'Connor. FUNCTION USE polymars polymars modeling predict.polymars fitted values from a polymars model plot.polymars plot a polymars model is.polymars is an object a object returned by polymars summary.polymars summarizes a polymars model and fitting steps polymars.persp used internally by plot.polymars SHAR_EOF fi if test -f 'plot.polymars' then echo shar: "will not over-write existing file 'plot.polymars'" else cat << \SHAR_EOF > 'plot.polymars' .BG .FN plot.polymars .TL plot.polymars - plots for polymars models .DN produces two and three dimensional plots of the fitted values from a model produced by `polymars'. .CS plot.polymars(polymars.model, predictor, response, predictor2, x, add=F, n, xyz=F, contour.polymars=F, intercept,...) .RA .AG polymars.model a model produced by the function `polymars'. .AG predictor the index of a predictor that was used when the polymars model was fit. For the two dimensional plots, this variable is plotted along the X-axis. .OA .AG predictor2 the index of a predictor that was used when the polymars model was fit. For the three dimensional plots, this variable is plotted along the Y-axis. See xyz. .AG response if the model was fitted to multiple response data the response index should be specified. By default response = 1. .AG x should be a vector of length equal to the number of predictors in the original data set. The values should be in the same order as in the original dataset. By default the function uses the median values of the data that was used to fit the model. Although the values for predictor and predictor2 are not used, they should still be provided. .AG add should the plot be added to a previously created plot? Works only for two dimensional plots. .AG n number of plotting points (2 dimensional plot) or plotting points along each axis (3 dimensional plot). The default is n = 100 for 2 dimensional plots and n = 33 for 3 dimensional plots. .AG xyz is the plot being made a 3 dimensional plot? If it is xyz should be set = T. If there is only one response it need not be set, if two numerical values accompany the model in the call they will be understood as two predictors for a 3-d plot. By default a 3-d plot uses the S persp function. Note: categorical predictors cannot be used for 3 dimensional plots. .AG contour.ploymars if the plot being made a 3 dimensional plot and contour = T a plot using the S function `contour' is being made. Note: categorical predictors cannot be used for 3 dimensional plots. .AG intercept is by default set = T so the fitted intercepts value in the model is included in the plots evaluation. Setting intercept = F evaluates a plot without the model's intercept. The intercept may also be given any numerical value which then overrides the fitted coefficient for the model. .AG ... Inherits arguments from the S functions `plot' and `persp'. .RT This function produces a 2-d plot of 1 predictor and response of a polymars fit at n equally spaced points or a 3-d plot of two predictors and response of a polymars fit. The range of the plot is by default equal to the range of the particular predictor(s) in the original data, but this can be changed by xlim=c(from,to) (and ylim=c(from,to)) .SA polymars, predict.polymars, summary.polymars .EX state.pm<-polymars(state.x77[,5],state.x77[,-5],gcv = 1) plot(state.pm,4,7) .WR SHAR_EOF fi if test -f 'polymars' then echo shar: "will not over-write existing file 'polymars'" else cat << \SHAR_EOF > 'polymars' .BG .FN polymars .TL Multi-response Multivariate Adaptive Regression Splines .DN A stepwise regression procedure using piecewise linear splines .CS polymars(responses, predictors, weights, maxsize, gcv=4, additive=F, startmodel, no.interact, knots, knot.space=3, ts.resp, ts.pred, ts.weights, classify, factors, tolerance, verbose=F) .RA .AG responses vector of responses, or a matrix for multiple response regression. In the case of a matrix each column corresponds to a response and each row corresponds to an observation. Missing values are not allowed. .AG predictors matrix of predictor variables for the regression. Each column corresponds to a predictor and each row corresponds to an observation in the same order as they appear in the response argument. Missing values are not allowed. .OA .AG weights vector of observation weights; if supplied, the algorithm fits to minimize the sum of the weights multiplied by the squared residuals. The length of weights must be the same as the number of observations. The weights must be nonnegative. .AG maxsize the maximum number of basis functions that the model is allowed to grow to in the stepwise addition procedure. The default is min(6*(number.of.observations^(1/3)),number.of.observations/4,100). .AG gcv parameter used to find the overall best model from a sequence of fitted models. The residual sum of squares of a model is penalized by dividing by the square of 1-(gcv x model size)/cases. A larger gcv value would tend to produce a smaller model. By default gcv is set to 4. .AG additive Should the fitted model be additive in the predictors? .AG startmodel the first model that is to be fit by the polymars function. It is either an object of the class polymars or a model specified by the user. In that case, it takes the form of a 4 x n matrix (n = number of basis functions in the starting model excluding the intercept term). Each row corresponds to one basis function (with two possible components). Column 1 is the index of the first predictor involved. Column 2 is a possible knot in this predictor. If column 2 is NA, the first component is linear. Column 3 is the possible second predictor involved (if column 3 is NA the basis function only depends on one predictor). Column 4 contains the possible knot for the predictor in column 3, and it is NA when this component is linear. Example: if a row reads 3 NA 2 4.7, the corresponding basis function is [X3 * (X2-4.7)_+]; if a row reads 2 4.3 NA NA the corresponding basis function is [(X2-4.3)_+]. A fifth column can be added with 1's and 0's, The 1's specify which basis functions of the startmodel must be in each model. Thus, these functions stay in the model during the whole stepwise fitting procedure. If no startmodel is specified polymars starts with a model that only contains the intercept. .AG no.interact a matrix used if certain predictor interactions are not allowed in the model. It is given as a matrix (2 * m), with predictor indices as entries. The two predictors of any row cannot have interaction terms with each other. .AG knots defines how the function is to find potential knots for the spline basis functions. This can be set to the maximum number of knots you would like to be considered for each predictor. Usually, to avoid the design matrix becoming singular the actual number of knots produced is constrained to at most every third order statistic in any predictor. This constraint can be adjusted using the `knot.space' argument. It can also be a vector with the number of potential knots for each predictor. Again the actual number of knots produced is constrained to be at most every third order statistic any predictor. A third possibility is to provide a matrix where each columns corresponds to the ordered knots you would like to have considered for that predictor. This matrix should be filled out to a rectangular data structure with NAs. The default is min(20,round(ncases/4)) knots per predictor. When specifying knots as a vector an entry of -1 indicates that the predictor is a categorical variable and each unique entry in it's column is treated as a level. When specifying knots as a single number or a matrix and there are categorical variables these are specified separately as such using the factor argument. .AG knot.space is an integer describing the minimum number of order statistics apart that two knots can be. Knots should not be too close to insure numerical stability. The default is 3. .AG ts.resp testset responses for model selection. Should have the same number of columns as the training set response. A testset can be used for the model selection. Depending on the value of classify, either the model with the smallest testset residual sum of squares or the smallest testset classification error is provided. Overrides gcv. .AG ts.pred testset predictors. Should have the same number of columns as the training set predictors. .AG ts.weights testset observation weights. A vector of length equal to the number of cases of the testset. All weights must be non-negative. .AG classify when the response is discrete (categorical), polymars can be used for classification. In particular, when classify = T, a discrete response with K levels is replaced by K indicator variables as response. Model selection is still being carried out using gcv, except when a testset is provided, in which case testset misclassification is used to select the best model. .AG factors used to indicate that certain variables in the predictor set are categorical variables. Specified as a vector containing the appropriate predictor indices (column numbers of categorical variables in predictors matrix). Factors can also be set when the "knots" argument is given as a vector, with -1 as the appropriate entries for factors. .AG tolerance for each possible candidate to be added/deleted the resulting residual sums of squares of the model, with/without this candidate, must be calculated. The inversion of of the "X-transpose by X" matrix, X being the design matrix, is done by an updating procedure c.f. C.R. Rao - Linear Statistical Inference and Its Applications, 2nd. edition, page 33. In the inversion the size of the bottom right-hand entry of this matrix is critical. If it's value is near zero or the value of it's inverse is almost zero then the inversion procedure becomes somewhat inaccurate. The lower the tolerance value the more careful the procedure is in selecting candidates for addition to the model but it may exclude too conservatively. And the other hand if the tolerance is set too high a spurious result with a singular or otherwise sub-optimal model may occur. By default tolerance is set to 1.0e-5. .AG verbose when set =T, the function will print out a line for each addition or deletion stage. For example, " + 8 : 5 3.25 2 NA" means adding interaction basis function of predictor 5 with knot at 3.25 and predictor 2 (linear), to make a model of size 8, including intercept. .RT The returned object contains information about the fitting steps and the model selected. The first data frame contains a row for each step of the fitting procedure. In the columns are: a 1 for an addition step or a 0 for a deletion step, the size of the model at each step, residual sums of squares (RSS) and the generalized cross validation value (GCV), testset residual sums of squares or testset misclassification, whatever was used for the model selection. The second data frame, model, contains a row for each basis function of the model. Each row corresponds to one basis function (with two possible components). The pred1 column contains the indices of the first predictor of the basis function. Column knot1 is a possible knot in this predictor. If this column is NA, the first component is linear. If any of the basis functions of the model is categorical then there will be a level1 column. Column pred2 is the possible second predictor involved (if it is NA the basis function only depends on one predictor). Column knot2 contains the possible knot for the predictor pred2, and it is NA when this component is linear. This is a similar format to the startmodel argument together with an additional first row corresponding to the intercept but the startmodel doesn't use a separate column to specify levels of a categorical variable . If any predictor in pred2 is categorical then there will be a level2 column. The column "coefs" (more than one column in the case of multiple response regression) contains the coefficients. The returned object also contains the fitted values and residuals of the data used in fitting the model. .DT is.polymars and summary.polymars are available. print.polymars defaults to summary.polymars. .SH REFERENCES Friedman, J. H. (1991). Multivariate adaptive regression splines (with discussion). The Annals of Statistics, 19, 1-141. Kooperberg, C., Bose, S. and Stone, C. J. (1997). Polychotomous Regression. Journal of the American Statistical Association, 92, - . .SA plot.polymars, predict.polymars, summary.polymars .EX state.pm<-polymars(state.region,state.x77,knots=15,classify=T) data<-na.omit(auto.stats) auto.pm<-polymars(data[1:40,1],data[1:40,-1],factor=c(2,3),ts.pred=data[41:66,-1],ts.resp=data[41:66,1],maxsize=40) .WR SHAR_EOF fi if test -f 'polymars.tex' then echo shar: "will not over-write existing file 'polymars.tex'" else cat << \SHAR_EOF > 'polymars.tex' \documentstyle[epsf]{article} \textwidth17cm \textheight22cm \hoffset-3cm \voffset-3cm \parskip=6pt \newcommand{\pic}[3] {\begin{figure}[ht] \begin{center} \caption{\label{#1}#3} \parbox{125mm}{ \epsfxsize=125mm \epsfysize=90mm \epsffile{#2.eps}} \end{center} \end{figure}} \newcommand{\pics}[3] {\begin{figure}[ht] \begin{center} \caption{\label{#1}#3} \parbox{125mm}{ \epsfxsize=125mm \epsfysize=60mm \epsffile{#2.eps}} \end{center} \end{figure}} \newcommand{\picl}[3] {\begin{figure}[ht] \begin{center} \caption{\label{#1}#3} \parbox{125mm}{ \epsfxsize=125mm \epsfysize=140mm \epsffile{#2.eps}} \end{center} \end{figure}} \begin{document} \large \title{POLYMARS} \author{Charles Kooperberg \and Martin O'Connor} \maketitle \section{Introduction} In this document we describe the POLYMARS procedure. POLYMARS is a variation on the Multiple Adaptive Regression Spline (MARS) procedure of Friedman (1991)\cite{F}, that was introduced by Kooperberg, Bose and Stone (1997)\cite{I}. In POLYMARS a multiple regression function is approximated using linear splines and their tensor products. POLYMARS can take multiple responses and can be used for classification. The POLYMARS procedure has a substantial number of options that make it a convenient tool for modeling. POLYMARS is implemented to be applicable to large data sets. \section{Description of POLYMARS}\label{van} \subsection*{The model} We assume that regression data is generated from a ``true model'' $$ Y = f(X_1,\ldots,X_p)+\epsilon.$$ In POLYMARS the function $f$ is approximated using functions that depend on only one or two of the $x_i$. That is, we use the model \[ f({\bf X}) = g_0+\sum_{j_1}g_{j_1}(X_{j_1})+\sum_{j_1decide which basis function can be added to the current model\\ \>add the best candidate\\ \>fit the model\\ \>evaluate the model\\ \>if the model is better than the best one, save it\\ \hspace*{.2cm} \}until(maximum model size is reached or no candidates are left)\\ do\{\\ \>decide which basis function can be removed from the model\\ \>remove the one that is the worst predictor\\ \>fit the model\\ \>evaluate the model\\ \>if the model is better than the best one, save it\\ \hspace*{.2cm} \}until(minimum model size)\\ \end{tabbing}} At each stage the procedure must find which potential basis functions can be added to the model. This procedure continues until a maximum model size is reached. (The maximum size of the model is a user specified parameter, the default of which depends on the sample size.) Then stepwise deletion of basis functions is employed. The candidates for removal are those that are currently in the model that can be removed taking a set hierarchical constraints, that are described in the next section, into account. As basis functions are added to the model the residual sum of squares gets smaller, no matter how unimportant the new basis function is. So, the residual sums of squares, while providing a good comparison for lack of fit between models of the same size, does not penalize for over-fitting. Instead we use a modified form of the cross-validation criterion originally proposed in Craven and Wahba\cite{C} and modified in Friedman and Silverman\cite{D} and Friedman\cite{E}. \begin{equation}\label{gcv} GCV(\hat{f}_M) = \frac{\frac{1}{n}\sum^N_{i=1}[y_i-\hat{f}_M(x_i)]^2} {\left [1-\frac{C(\hat{f}_M)}{n}\right ]^2}, \end{equation} where $C(\hat{f}_M)$ is a function proportional to the number of basis functions added, $C(\hat{f}_M)$ = $d \times M$ Here $d$ is a constant that penalizes for larger models, usually $3\leq d \leq 5$. This criterion is evaluated at the end of each addition and deletion step and the best-model-so-far by this criterion is stored. In our implementation this constant $d$ is referred to as {\tt gcv} and can be specified by the user. \subsection*{Which candidate functions considered for model selection} At the initial (addition) stage a constant model is fit. After that, the number of basis functions that are candidates for addition depends on the number of possible knots per predictor. With more than a few predictors and knots this can be a large number, so evaluating each possible candidate is the computationally most expensive chore of POLYMARS. To keep the procedure fast we limit the possible candidates we use at each addition step. We specifically take these rules, since this procedure will prefer univariate functions and linear functions, which will yield a better interpretable final model. The candidate basis functions are: \begin{itemize} \item $x_i$, $i=1,\ldots,p$; \item $(x_i-t_{ik})_+$ if $x_i$ is already a basis function in the model; \item $x_ix_j$ if $x_i$ and $x_j$ are already basis functions in the model; \item $x_i(x_j-t_{jk})_+$, if $x_ix_j$ and $(x_j-t_{jk})_+$ are in the model; \item $(x_i-t_{ik})_+ (x_j-t_{jk})_+$ if $x_i(x_j-t_{jk})$ and $x_j(x_i-t_{ik})$ are in model. \end{itemize} The number of candidate knots is a user specified parameter. Initially the procedure computes order statistics for each predictor and take a subset of these as knots. Typically we may consider 20 or so potential knots for a predictor. Kooperberg, Bose, and Stone (1997) \cite{I} argue that by limiting the number of candidate knots, the resulting procedure will be an order of magnitude faster than MARS. The first iteration would fit a linear basis function to one of the predictors and the second iteration would consider linear basis functions in one of the other predictors as well as basis functions with knots on the predictor already added. This creates a preference in the procedure for linear models over nonlinear ones, while interactions are only considered if they are between predictors that are already in the model. For the deletion stage, the candidates for removal are the basis functions in the model that, when removed, yield a valid remaining model. Each of these basis functions is taken out in turn and the {\em RSS} for each reduced model is calculated. The model which results in the smallest increase in RSS becomes the new model. \section{Options of POLYMARS}\label{adds} \subsection*{Multiple Response Regression} When the response for each observation is a vector, we can use a multiple response version of POLYMARS. The model fitted uses the same basis functions for each response but has different coefficients. The coefficients are computed as in equation~(\ref{matCOEF}) $${\bf \hat{\beta}} = ({\bf X}^T{\bf X})^{-1}{\bf X}^T{\bf Y}$$ The usual expression for the RSS \[RSS = Y^TY - Y^T{\bf X}\hat{\beta}, \] now becomes $$RSS = {\bf Y}^T{\bf Y} - {\bf Y}^T{\bf X}{\bf \hat{\beta}}.$$ The responses, ${\bf Y}$ and the coefficients ${\bf \hat{\beta}}$ are matrices. \subsection*{Model selection using a test set} A test data set may be used to select the best overall model. The models fitted in the stepwise addition and deletion stages uses the original (training) data. The overall best model for these is then selected by how well it fits the test data by a lack of fit criteria, usually $RSS$. \subsection*{Using POLYMARS as a classifier} As a multiple response regression procedure POLYMARS can be used as a classifier, see Kooperberg, Bose and Stone~\cite{I}. If we have a vector $Y$ that can take on a finite number of discrete values (classes) it may be appropriate to fit POLYMARS model to predict the corresponding classes based on a vector of covariates ${\bf X}^\prime$. Rather than fitting a multiple classification model using a polychotomous likelihood, we use POLYMARS with multiple response regression. Here for each case ${\bf Y}$ is the binary vector of length $C$ with one 1, corresponding to the correct class. A numerical response vector can be given with the argument {\tt classify = T} and the procedure will convert this response to multiple binary columns ($Y$ should contain only integers). If $Y$ is a vector of characters is always considered a classification problem ({\tt classify = T} need not be set). For classification by POLYMARS with a training set and test set the best model is selected using a loss function with unit loss for misclassification on the test set, adjusted for model-size similar to how we deal with $RSS$ in equation~\ref{gcv}. \subsection*{Weights can also be supplied for the training and test sets.} \section{Using POLYMARS}\label{using} \subsection{Arguments to POLYMARS} See the help files for more details. \begin{itemize} \item {\tt responses} and {\tt predictors} The only required arguments needed for the POLYMARS function. \item {\tt weights} Case-weights, nonnegative. \item {\tt maxsize} The maximum number of basis functions that the model can grow to. This will not be reached if no more candidate basis functions can be fit to the model at any addition step with model-size less than {\tt maxsize}. By default maxsize = $\min(6\times n^{\frac{1}{3}},\frac{n}{4},100)$ where $n$ is the number of cases in the dataset. \item {\tt no.interact} For interpretability it may be desired that certain interactions should not be allowed. A $2\times m$ matrix of these disallowed interaction terms is given to the {\tt no.interact} argument. Each row contains the indices of a pair of predictor variables that may not contain interaction terms in the model. The procedure doesn't normally support interactions involving categorical variable levels, see {\tt factors} below for more about this. \item {\tt additive} Setting {\tt additive = T} requires that the procedure only considers additive models. No interaction term will be considered for the model. \item {\tt startmodel} \nopagebreak An initial model may be specified. This has three main purposes. It suggests a starting point in model selection for datasets where there are many possible predictors but a small subset may be considered more important than the others. Further it may specify basis functions that the user feels {\em must} be in the model. Using {\tt startmodel}, basis functions can be specified to be in all models considered. Because of the order in which basis functions must be added for any one predictor the procedure may be adverse to fitting the first linear term to predictors with a highly curved relationship which is almost orthogonal to the predictor itself. In this case a linear basis function may improve the {\it RSS} of the model very little beyond the intercept although two basis functions would fit well. By introducing an initial linear term in the {\tt startmodel} such predictors have a better chance of being in the final model and if they are inappropriate the procedure should remove them at the deletion stage. See the help files for the required format of {\tt startmodel}. \item {\tt knots} Crucial to the procedure in modeling non-linearity are the possible knots for the basis functions. Ideally every data point for a predictor should be considered as a knot location. However due to computational expense it is usually better to consider only a subset of the data points. See the help files for the possible formats of {\tt startmodel}. \item {\tt knot.space} As noted in the section on the {\tt knots} argument, the knots are selected as evenly spaced ordered statistics of the original data points, constrained to be at most every third order statistic. This is a numerical stability safeguard as basis functions with knots too close in the model may cause the design matrix to be nearly singular and unstable. This can be changed with the {\tt knot.space} argument. \item {\tt factors} Categorical predictors are converted to dummy zero-one variables so a factor of $k$ levels can have at most $k-1$ of its levels in the model. The {\tt factors} argument specifies which predictor variables are to be treated as categorical. All unique values in the data-set for that predictor are then treated as levels. The procedure doesn't support interactions involving categorical variables. This can be gotten around by some pre-processing of the data, converting categorical variables into dummy $0/1$ variables not defined as categorical. \item {\tt classify} For a classification problem the responses can be treated as classes instead of a continuous variable. If the response is a vector of characters the procedure will automatically perform $POLYMARS$ with classification. If the response is of integer form this is done when {\tt classify = T} is given as an argument. See Section~\ref{adds}. If {\tt ts.resp} and {\tt ts.pred} are given as arguments, the rate of misclassification of the test set response will be used to select the overall model for the best model at each step. It is adjusted for model-size as $RSS$ is. Candidate basis functions are still evaluated by $RSS$. \item {\tt ts.resp} and {\tt ts.pred} At each step a candidate is selected for addition or deletion and a new larger or smaller model is then selected. Normally a $GCV$ score based on the original training dataset is computed, see Section~\ref{van}, so as to compare the model selected at each step adjusting for model size. When {\tt ts.resp} and {\tt ts.pred} are arguments, the $RSS$ of the test set is used to find the best overall model from the models produced at each addition and deletion stage. \item {\tt ts.weights} Case-weights for the test set, nonnegative. \item {\tt tolerance} We also take advantage of the fact that $X^TX$ grows only by adding a row and column when we compute $(X^TX)^{-1}$ after one addition step. By storing the $(X^TX)^{-1}$ after each addition step the new $(X^TX)^{-1}$ from an $X$ matrix with one more column can be computed without direct matrix inversion. Using a result from Rao\cite{B} p.33 :- When A and D are symmetric matrices and all inverses exist $$ \left( \begin{array}{ll} A & B\\ B^T & D \end{array} \right)^{-1} = \left( \begin{array}{ll} A^{-1}+FE^{-1}F^T & -FE^{-1}\\ - E^{-1}F^T & E^{-1} \end{array} \right) $$ where $E = D - B^TA^{-1}B$ and $F = A^{-1}B$. The only inverses needed are $A^{-1}$ which is the $(X^TX)^{-1}$ of the model produced by the previous iteration and $D^{-1}$ which in our case is a scalar ($ = \sum B_i(x_i)^2$ for the new basis fit). So the initial $(X^TX)^{-1}$ is a scalar and it is built on to get all the following $(X^TX)^{-1}$s. The B and D components are gotten from the candidate part of the $Y^TX\mid X^TX$ matrix. This matrix inversion must be done for each candidate at each step of the procedure and so benefits greatly from being fast. Normally inverting an $r \times r$ matrix requires 0($r^3$) operations but here it is reduced to 0($r^2$). So $Y^TX$ and $(X^TX)^{-1}$'s main components are stored from iteration to iteration. The $X$ matrix is also stored and updated for calculating inner products with new candidates. This procedure is reversed during the deletion stage of the algorithm. It has been found that the size of the element $E$ which is a number in our case is important for stability. Basis functions that cause the design matrix to become unstable (nearly singular) produce either very large or very small values for $E$. To avoid instability of the numerical procedure, (computing $(X^TX)^{-1}$), candidate basis functions are rejected if they produce values of $E$ less than the value of {\tt tolerance} or greater than 1 divided by {\tt tolerance}. The default of {\tt tolerance} is $1.0\times 10^{-5}$. {\bf Note} This implementation uses a matrix inversion function from the $Lapack$ libraries. This function is called if there is an initial model specified and once per addition/deletion step to update the $(X^TX)^{-1}$ matrix. \item {\tt verbose} When {\tt verbose = T} is given as an arguement some information is printed during the model fitting. A {\tt +} means an addition to the model, a {\tt -} means a basis function is taken away. The model size is printed next and the basis function which is added or deleted. \end{itemize} \subsection{Other functions} See the help files for details \begin{itemize} \item{\bf plot.polymars} Plots 1 and 2 predictor graphs of a model returned from polymars. \item{\bf predict.polymars} Computes fitted values of a model returned from polymars. \item{\bf print.polymars, summary.polymars} Prints formatted output of a model returned from polymars. \end{itemize} \subsection{Interpreting the returned model} \begin{itemize} \item {\tt mymodel<-polymars(theresponse,thepredictors)} \item The visible model\\ Using {\tt summary} or {\tt print} (which defaults to {\tt summary}) on an object returned from {\tt polymars} prints out three components of the POLYMARS object ``mymodel''. \begin{itemize} \item The {\bf call} which produced the object. \item An Splus data-frame, {\bf fitting} which contains certain statistics about the model fitting routine. Each row represents one step in the fitting routine. The first column {\em 0/1} has a $1$ for an addition step and a zero for a deletion step. The second column, {\em size}, contains the resulting number of basis functions in the model after this step. Next there is a column {\em RSS} containing the residual sum of squares. For multiple response regression or classification there will be more than one column {\em RSS1}, {\em RSS2}$\ldots$, one for each response or class. The last column contains the measure of fit which used to find the best overall model. It is normally {\em GCV} for generalized cross validation, see Section~\ref{van}. But it can also be {\em T.S. RSS} for test set residual sum of squares or {\em T.S.M.C.} for test set misclassification, see Section~\ref{adds}. \item The POLYMARS {\bf model} itself is printed as a data-frame, each row corresponding to a basis function. The first row corresponds to the intercept. The first four (or five) columns relate to the basis function and the last column (more than one for multiple response regression or classification) contains the coefficients. For classification coefficients see also {\em conversion} below. The first column {\em pred1} contains the index of the first predictor of the basis function. Column {\em knot1} is a possible knot in this predictor. If this column is NA, the first component is linear. If any of the basis functions of the model are categorical then there will be a {\em level1} column. Column {\em pred2} is the possible second predictor involved (if it is NA the basis function only depends on one predictor). Column {\em knot2} contains the possible knot for the predictor {\em pred2}, and it is NA when this component is linear. For example the following model \begin{verbatim} pred1 knot1 level1 pred2 knot2 coefs SE 1 0 NA NA 0 NA 59.123913 9.0277 2 13 NA NA 0 NA -5.508833 2.2838 3 13 6.29 NA 0 NA 4.834013 1.9678 4 4 NA 1 0 NA 9.486980 2.6050 \end{verbatim} has an intercept of $59.12$ and predictor 13 has two terms in the model, a linear term with coefficient $-5.51$ and a term with a knot $(X13-6.29)_+$ with coefficient $4.83$. One level of variable 4 is in the model (this was actually a 0/1 variable) with coefficient $9.49$. Standard errors for the coefficients are also included. A line such as \begin{verbatim} pred1 knot1 pred2 knot2 coefs SE 2 0 3 0.28 12.45667 3.3382 \end{verbatim} corresponds to a basis function $X2\times(X3-0.28)_+$ with coefficient 12.46. \end{itemize} \item The invisible attributes \begin{itemize} \item {\bf model.size} contains the number of basis functions in the returned model. \item {\bf fitted} and {\bf residuals} contain the fitted values and the residuals of the original dataset used to fit the model. \item {\bf responses} contains number of responses per case in original dataset. \item {\bf ranges.and.medians} is a $3\times p$ matrix, where $p$ is the number of predictors in the original dataset, each column corresponding to a predictor. Rows 1 and 2 contain the minimum and maximum values of the predictor and row 3 contains the median value of the predictor. These are computed from the original (training) dataset and are used in the plotting function. \item {\bf factor.matrix} is is a $r \times s$ matrix where $r$ is the number of categorical predictor variables in the model and $s$ is the maximum number of levels that any of these has $+2$. Each column represents a categorical variable. The first row contains the index of the predictor, the second row contains the number of levels it has and the remaining rows contain the value of each level (filled out to the end with NA's if necessary). This is used in the plotting function. \item {\bf conversion} is a $c \times 2$ matrix where $c$ is the number of classes or categories in the response when POLYMARS is used as a classifier. In classification a single response vector is split up into a multiple response matrix with each column corresponding to a class and each case having a 1 in the column corresponding to it's original response and all other columns zero. In the {\bf conversion} matrix each row corresponds to one class. In the first column it's original class, as a character or number, is recorded. The second column holds the response number, or column index of the new response matrix. The coefficients of {\bf model} are ordered according to this numbering. It is used for computing further fitted values (classes). \end{itemize} \end{itemize} \section{Examples} \subsection{Boston housing data} The data for this example comes from Statlib, ({\tt http://lib.stat.cmu.edu/}). The data-set contains variables on various criteria such as distance to urban amenities, pollution and crime which may effect house prices. The data is originally from Harrison and Rubinfeld~\cite{H}. The variables in this data set are: \begin{tabular}{lll} 1. && per capita crime rate by town.\\ 2. && proportion of residential land zoned for lots over 25,000 sq.ft.\\ 3. && proportion of non-retail business acres per town.\\ 4. && Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\\ 5. && nitric oxides concentration (parts per 10 million)\\ 6. && average number of rooms per dwelling\\ 7. && proportion of owner-occupied units built prior to 1940\\ 8. && weighted distances to five Boston employment centers\\ 9. && index of accessibility to radial highways\\ 10.&& full-value property-tax rate per \$10,000\\ 11.&& pupil-teacher ratio by town\\ 12.&& $1000(Bk - 0.63)^2$ where Bk is the proportion of blacks by town\\ 13.&& \% lower status of the population\\ 14.&& Median value of owner-occupied homes in \$1000's\\ \end{tabular} For this regression we left all variables untransformed. The response variable is $x_{14}$, the median value of owner-occupied homes in \$1000's. For demonstration we asked for 15 possible knots per predictor, some predictors will have less than this, for example ($x_2$ has 73\% of it's cases equal to zero) so will have less knots. Variable 4, $x_4$, is specified as a categorical variable (this is a bit redundant as it only has two levels). The procedure will use a test set and training set instead of the usual {\em GCV} parameter to select the model so we split the data into two groups. For interpretation of the final model we may want to exclude certain possible interactions such as a nitrous oxide/rooms per dwelling effect. In this example we picked two interactions that we didn't want to see in the model, those between $x_5$ and $x_6$ and between $x_3$ and $x_{11}$. For a realistic analysis there are other interactions which are just as uninterpretable. \noindent The {\it Splus} code was\\ {\tt index<-sample(nrow(boston.dat),nrow(boston.dat)/2) \noindent dni<-matrix(c(5,6,3,11),byrow=T,ncol=2) \noindent boston.mars<-polymars(boston.dat[index,14], boston.dat[index,-14],\\ \hspace*{2in}knots=15,factor=4, ts.resp=boston.dat[-index,14],\\ \hspace*{2in}ts.pred=boston.dat[-index,-14], no.interact=dni) } \noindent The returned model was: \begin{tabular}{lr}\hline Basis function & Coefficient \\\hline Intercept & -115.92\\ $x_1$ & -0.18\\ $x_6$ & 31.10\\ $(x_6-6.44)_+$ & 5.94\\ $(x_6-7.84)_+$ &-9.37\\ $x_8$ &-43.72\\ $(x_8-1.53)_+$ &43.07\\ $x_{11}$ &11.34\\ $(x_{11}-17.80)_+$ &-2.02\\ $x_{13}$ &-1.21\\ $(x_{13}-5.52)_+$ &-1.98\\ $x_6\times x_{11}$ &-1.56\\ $x_8\times x_{13}$ &0.04\\ $x_{11}\times x_{13}$ &-0.07\\ \hline \end{tabular} \noindent This model has a multiple $R^2$ of 0.93. \subsection{A simple function of ten variables} This example is taken from Friedman's MARS~\cite{A} paper which as an example illustrates the ability of the procedure to find structure is data while ignoring noise. The data is created using the function $$ f({\bf x}) = 10\sin(\pi x_1x_2)+20(x_3-\frac{1}{2})^2+10x_4+5x_5+ \epsilon$$ in a $n = 10$ dimensional hypercube using $N = 100$ points. Covariates were drawn from a uniform distribution and $\epsilon$ is from a standard normal distribution. Figure~\ref{a} shows the original function without noise for variables $x_1$, $x_2$ and $x_3$. $f({\bf x})$ is linear in $x_4$ and $x_5$. This example with it's highly curved relationships is not an ideal setting for POLYMARS with it's piecewise linear splines. This is particularly true for $x_3$ as our procedure insists that a linear basis function is the first term fit to any variable. A linear term will not improve the fit any more than just the constant intercept term so the model building procedure will be averse to adding any $x_3$ terms. For this reason we specify $x_3$ to be in the startmodel. The model with $gcv$ equal to 3.0 and the default number of knots, $knots = 20$. \noindent The {\it Splus} code was\\ {\tt mars2 <- polymars(responses = yy, predictors = cbind(x1, x2, x3, x4, x5, x6, x7, x8, x9, x10), gcv = 3, startmodel = c(3, NA, 0, 0))} \newpage \noindent The returned model was \begin{tabular}{lr}\hline Basis function & Coefficient \\\hline Intercept & -2.21\\ $x_1$ & 13.83\\ $x_2$ & 13.18\\ $(1 - 0.71)_+$ & -23.50\\ $(2 - 0.55)_+$ & -12.16\\ $x_3$ & -10.18\\ $(3 - 0.45)_+$ & 17.94\\ $x_4$ & 9.37\\ $x_5$ & 4.69\\ \hline \end{tabular} Figure~\ref{b} shows the fitted functions for $x_1$, $x_2$ and $x_3$. The coefficients of $x_4$ and $x_5$ are quite close to that of the original function. The procedure works quite well in picking up the main structure present in the data without picking up any of the 5 noise covariates. The introduction of an initial linear term to detect {\em U-shaped} relationships is something that would be explored in exploratory data analysis. Calling POLYMARS with a linear term for all 10 variables in {\tt startmodel} may be also a reasonable thing to do. The maximum size the model should grow to ({\tt maxsize}), was set to $35$ to compensate for these initial basis functions being already in the model (by default the maximum size is $25$ for this size of a dataset, see section~\ref{using}). The model produced was very similar to the one above with one extra basis function, $x_6$. This had a coefficient of $-2.43$. It did pick up one of the noise variables but regressing the $x$ variables on $y$ using {\em Splus} function {\tt lm} fits a coefficient of $-3.35$ on $x_6$ with a t-value of $-3.37$ with $89$ degrees of freedom, the most ``significant'' of the noise variables. This example was original chosen to demonstrate the capabilities of a different procedure, but this piecewise linear $MARS$ implementation works quite well. \subsection{Phoneme data} The source of this data is the Center for Spoken Language Understanding in Portland, Oregon. A full description of the data can be found in Kooperberg and Stone~\cite{G}. The data used in this example is a subset of the original dataset consisting of a three class response, corresponding to three phonemes; the vowels in b{\em ee}t, b{\em e}t and b{\em ou}ght. The predictor set consists of 81 variables. These variables were obtained by processing the audible spectrum of the utterance that produced each phoneme, to produce perceptual linear predictive (PLP) features. The dataset consists of a training set of 14,735 cases and a test set of 10,167 cases. We selected a random subset of 1000 cases from the training set and 300 from the test set although this implementation has also be used on the full set. \newpage \noindent The model produced contained 17 of the predictor variables summarized as follows:\\ \begin{tabular}{lr}\hline Intercept & \\ Linear variables & 9\\ Variables with 1 knot & 5\\ Variables with 2 knots & 1\\ Variables with 3 knots & 1\\ Interactions with 2 linear terms & 12\\\hline Total number of basis functions (with intercept)& 39 \end{tabular} \noindent Fitted values were obtained from this model for the remaining 13,736 cases. The model had a misclassification rate of 6\% on this data. \noindent The {\em Splus} code was\\ {\tt index1<-sample(1:14736,1000)\\ index2<-sample(1:10617,300) \noindent phoneme.mars<-polymars(phontr.dat2[index1,1],phontr.dat2[index1,-1],\\ ts.resp=phonts.dat[index2,1],ts.pred=phonts.dat[index2,-1],\\ knots=30,classify=T) \noindent predictions<-predict.polymars(phoneme.mars,phontr.dat[-index1,-1],classify=T) } \section*{Appendix: Outline of the program} The basic program flow \begin{enumerate} \item {\bf Mesh computed} If the possible knots for each predictor are not given as an argument knots are calculated for each predictor. They as selected as evenly spaced order statistics. \item{\bf Initial model fit} If an initial model is an argument to polymars, it is checked for consistency to the addition of terms ordering and then fit. Any knots involved are added to the mesh. If no initial model is specified an intercept is fit. \item{\bf Addition iteration} \begin{enumerate} \item{All possible (new) candidates are found and stored.} \item{The best of these candidates is found} \item{The best of candidate is added to the model and removed from the candidates. If no candidate can be fit, addition stops.} \item{If largest model size specified has been reached, addition to the model stops.} \end{enumerate} \item{\bf Deletion iteration} \begin{enumerate} \item{From all possible candidates for removal from the model, the best one is calculated} \item{The best candidate for removal is taken out of the model} \item{If the smallest possible size is reached the deletion stage stops} \end{enumerate} \item{\bf Best model is reported} \end{enumerate} \begin{thebibliography}{99} \bibitem{C} Craven, P., and Wabha, G. (1979), Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation, {\em Numerical Mathematics}, {\bf 31}, 317--403. \bibitem{E} Friedman, J.H. (1988), Fitting functions to noisy data in high dimensions. In {\em Computing Science and Statistics: Proceedings of the Twentieth Symposium on the Interface}, (E.J. Wegman, D.T. Gantz and J.J.Miller, eds.) 13--43. American Statistical Association, Alexandra, Virginia. \bibitem{A} Friedman, J.H. (1991), ``Multivariate Adaptive Regression Splines (with discussion),'' {\em The Annals of Statistics}, {\bf 19}, 1--141. \bibitem{D} Friedman, J.H., and Silverman, B.W. (1989), ``Flexible parsimonious smoothing and additive modeling,'' {\em Technometrics}, {\bf 31}, 3--39. \bibitem{H} Harrison, D. and Rubinfeld, D.L. (1978), ``Hedonic prices and the demand for clean air,'' {\em J. Environ. Economics \& Management}, {\bf 19}, 81--102. \bibitem{F} Hastie, T.J., and Tibshirani, R.J. (1994), {\em Generalized Additive Models}, Chapman \& Hall. \bibitem{I} Kooperberg, C., Bose, S., and Stone C.J. (1997), ``Polychotomous Regression'', {\em Journal of the American Statistical Association}, {\bf 92}, 117--127. \bibitem{G} Kooperberg, C., and Stone C.J. (1997), ``Using the Stochastic Gradient Method to fit Polychotomous Regression Models'', {\em University of Washington, Department of Statistics, Technical Report} 319. \bibitem{B} Rao, C. R. (1989), {\em Linear Statistical Inference and Its Applications}($2^{nd}$ ed.), Wiley \& Sons. \bibitem{S} Stone, C. J., Hansen, H., Kooperberg, C., and Truong, Y. K. (1997), ``The use of polynomial splines and their tensor products in extended linear modeling (with discussion),'' {\em The Annals of Statistics}, to appear. \end{thebibliography} \pic{a}{fig1}{Original function without noise}%..................................................... \pic{b}{fig2}{POLYMARS fit of the noisy data}%..................................................... \end{document} SHAR_EOF fi if test -f 'polymarsall.S' then echo shar: "will not over-write existing file 'polymarsall.S'" else cat << \SHAR_EOF > 'polymarsall.S' # Time-stamp: <97/07/08 16:02:17 martin> #****************************************************************************** # (c) Charles Kooperberg and Martin O'Connor 1997 * # This function is part of an implementation of the Multivariate Adaptive * # Regression Splines (MARS) methodology, first proposed by J.H. Friedman(1991)* # The Annals of Statistics, 19, 1 - 141. * # The program is a modified version of MARS (PolyMARS) described in * # Kooperberg, C., Bose, S. and Stone C.J. (1997) ``Polychotomous Regression'',* # 92, 117 - 127. * # Journal of the American Statistical Association * # You are free to use this program, for non-commercial purposes only, * # under the condition that * # this note is not to be removed. * # * # The program is not formally maintained, but we are interested in hearing * # from people who have problems with it, although we may not be able to solve * # them. * # Email clk@fhcrc.org or martin@stat.washington.edu * # Charles Kooperberg and Martin O'Connor, May 20, 1997 * #*****************************************************************************/ polymars<-function(responses,predictors,maxsize,gcv=4.0,additive=F,startmodel,weights,no.interact,knots,knot.space=3,ts.resp,ts.pred,ts.weights,classify,factors,tolerance=1e-5,verbose=F) { #responses - a vector (or matrix) of responses. (Can be a a vector of characters for classification) #predictors - a matrix of predictors with same number of cases as response. Columns are predictors. #OPTIONAL ARGUEMENTS #maxsize - maximum number of basis function the model can contain #gcv - parameter for overall best model seletion #additive - boolean, is the model to be additive #startmodel - either a matrix (m*4 or m*5) or a polymars object from a previous call to polymars # an initial model the procedure should start with in model selection #weights - a vector of length equal to the number of cases #no.interact- a 2*l matrix of columns numbers of the predictor matrix( each row pair cannot # have interaction terms) #knots - a vector specifying many knots per predictor are wanted (with -1 for categorical # variables) ncol(predictors)==length(knots), or a matrix with ncol(predictors) == # ncol(knots) with actual knot specified and filled out with NA's. # Can also be a single number - "knots" number of knots per predictor #knot.space - minimum number of order statistics between knots #ts.resp - testset reponses, same format as responses #ts.pred - testset predictors, same format as predictors #ts.weights - testset weights, same format as weights #classify - whether classification is to be done, set = T if the response vector is integer, if # if character classify is automatically true #factors - a vector of column numbers of the predictor matrix of categorical variables #tolerance - a numerical parameter which may need to be made smaller if the program crashes #store the call to the mars function call <- match.call() if(!is.loaded(symbol.C("polymars"))) dyn.load.shared("/shou6/users/martin/ra/mars/clean/polymars.so") responses<-as.matrix(responses) predictors<-data.matrix(predictors) nresponses<-ncol(responses) npredictors<-ncol(predictors) ncases<-nrow(predictors) if(missing(classify))classify<-F if(mode(responses)=="character" || classify == T) { if(ncol(responses) > 1) { stop("When using character responses or classify = T only 1 response per case is allowed\n") } char.responses<-responses int.responses<-as.integer(as.factor(responses)) nresponses<-length(unique(responses)) responses<-matrix(ncol=nresponses,nrow=ncases,data=int.responses) for(i in 1:nresponses) { responses[,i]<-(responses[,i] == (unique(int.responses)[i])) } conversion <- matrix(ncol=2,nrow=nresponses,c(unique(char.responses),unique(int.responses))) classify<-T if(!missing(ts.resp)) { char.responses.test<-ts.resp ts.resp<-matrix(ncol=nresponses,nrow=length(char.responses.test),data=0) for(i in 1:nresponses) { ts.resp[,i]<- as.integer(char.responses.test == conversion[i,1]) } } } else { conversion <- F classify<-F } #maxsize that the model can grow to if(missing(maxsize))maxsize<-ceiling(min(6*(ncases**(1/3)),ncases/4,100)) #if a testset is to be used in model selection if(!missing(ts.resp) || !missing(ts.pred)) { if(missing(ts.resp) || missing(ts.pred)) { stop("Both ts.resp (testsets responses) and ts.pred (testset predictors) should be specified\n") } if(!is.matrix(ts.resp))ts.resp<-as.matrix(ts.resp) if(!is.matrix(ts.pred))ts.pred<-as.matrix(ts.pred) if(ncol(ts.resp) != nresponses) { stop("Testset should have the same number of responses as the training set\n ") } if(ncol(ts.pred) != npredictors) { stop("Testset should have the same number of predictors as the training set\n ") } if(nrow(ts.resp) != nrow(ts.pred)) { stop("Testset ts.pred and ts.resp should have the same number of cases (rows)"); } testsetmatrix<-cbind(ts.resp,ts.pred) testsetcases<-nrow(testsetmatrix) testset<-T if(!missing(ts.weights)) { if(length(ts.weights) != testsetcases) { stop("length of testset weights misspecified\n") } testset.weighted<-T } else { testset.weighted<-F ts.weights<-0 } } else { testsetmatrix<-0 testsetcases<-0 testset<-F testset.weighted<-F ts.weights<-0 } #If the mesh is specified by the knots arguement this will be changed to #true later mesh.specified<-F mesh.vector<-0 if(nrow(responses) != nrow(predictors)) { stop("The number of rows (cases) of the response and predictor matricies should be the same") } if(!missing(knots) && !is.matrix(knots) && length(knots) != npredictors && length(knots) !=1) { stop("Length of vector of `knots per predictor' should be equal to number of predictors or 1\n") } if(!missing(knots)) { if(!is.matrix(knots)) { #if knots is specified as a single number it is expanded to a vector #length npredictors if(length(knots) ==1){knots<-rep(knots,npredictors) if(!missing(factors)) { for(i in 1:length(factors)) { if(! is.vector(factors)) { stop("`factors' should be a vector whose elements are indicies of predictors that are factors\n") } # in knots the number of knots(per predictor) is specified # or -1 if the predictor is a factor and all it values are levels knots[factors[i]] <- -1 } } } } else { mesh<-knots mesh.vector<-vector(length=ncol(mesh)*nrow(mesh),mode="double") knots<-vector(length=npredictors,mode="integer") k<-0 for(i in 1:npredictors) { knots[i]<-length(unique(mesh[is.na(mesh[,i])==F,i])) for(j in 1:knots[i]) { k<-k+1 mesh.vector[k]<-unique(mesh[!is.na(mesh[,i]),i])[j] } } if(!missing(factors)) { for(i in 1:length(factors)) { if(! is.vector(factors)) { stop("`factors' should be a vector whose elements are indicies of predictors that are factors\n") } # in knots the number of knots(per predictor) is specified # or -1 if the predictor is a factor and all it values are levels knots[factors[i]] <- -1 } } mesh.specified <-T } } if(missing(knots)) { knots<-rep(min(20,round(ncases/4)), npredictors) if(!missing(factors)) { for(i in 1:length(factors)) { if(! is.vector(factors)) { stop("`factors' should be a vector whose elements are indicies of predictors that are factors\n") } # in knots the number of knots(per predictor) is specified # or -1 if the predictor is a factor and all it values are levels knots[factors[i]] <- -1 } } } startmodelsize<-1 #A starting model must be specified as a object of class polymars #or a matrix with 4 or 5 columns no.remove<-0 no.remove.size<-0 if(!missing(startmodel)) { if(is.vector(startmodel))startmodel<-t(as.matrix(startmodel)) if(!(is.matrix(startmodel) || is.polymars(startmodel)) || (is.matrix(startmodel) && (ncol(startmodel) != 4 && (ncol(startmodel) != 5)))) { stop("startmodel should be a matrix with each row corresponding to a function with number of columns = 4 (or 5 for extra boolean column specifying predictors which cannot be removed \n") } if(is.matrix(startmodel)) { #Fifth column denotes which basis functions must remain in the model at #all times if(ncol(startmodel) == 5) { no.remove<-vector(length=(nrow(startmodel))) j<-0; for(i in 1:nrow(startmodel)) { if(startmodel[i,5]==T) { j<-j+1 no.remove[j]<-i } } no.remove.size<-j } #The startknots are taken from the startmodel and put into a vector #The startmodel becomes a 4*n matrix with a "1" in the 2nd and 4th #columns where knots appear startknots<-as.vector(t(cbind(startmodel[,2],startmodel[,4]))) startknots[is.na(startknots)]<-0.0 startmodel<-matrix(startmodel[,1:4],ncol=4) startmodel[!is.na(startmodel[,2]),2]<-1 startmodel[is.na(startmodel[,2]),2]<-0 startmodel[is.na(startmodel[,3]),3]<-0 startmodel[startmodel[,3]==0,4]<-0 for(i in 1:nrow(startmodel)) { if((!is.na(startmodel[i,4])) && startmodel[i,3]!=0) startmodel[i,4]<-1 } startmodel[is.na(startmodel[,4]),4]<-0 startmodelsize<-nrow(startmodel)+1 } else { startmodelsize<-startmodel$model.size startmodel<-startmodel$model[-1,] startknots1<-startmodel$knot1 startknots2<-startmodel$knot2 L1<-F if(!is.null(startmodel$level1)) { L1<-T level1<-startmodel$level1 } if(L1) { startmodel$knot1[!is.na(level1)]<-1 startknots1[!is.na(level1)]<-level1[!is.na(level1)] } startknots<-cbind(startknots1,startknots2) startknots<-as.vector(t(startknots)) startknots[is.na(startknots)]<-0.0 startmodel<-cbind(startmodel[,"pred1"], startmodel[,"knot1"], startmodel[,"pred2"], startmodel[,"knot2"]) startmodel[,2]<-!is.na(startmodel[,2]) startmodel[,4]<-!is.na(startmodel[,4]) } } else { startmodel<-0 startknots<-0 } if(!missing(weights)) { if(length(weights) != ncases) { stop("Number of weights not equal to the numnber of cases\n") } weighted<-T } else { weighted<-F weights<-0 } datamatrix<-cbind(responses,predictors) #Predictors which cannot interact together in the model are specified #by a 2*n matrix of predictor indicies if(!missing(no.interact)) { if(!is.matrix(no.interact) || ncol(no.interact)!=2) { stop("list of interactions disallowed has been misspecified,must be a 2*n matrix") } no.interact<-t(no.interact) no.interact.size<-ncol(no.interact) } else { no.interact.size<-0 no.interact<-0 } if(startmodelsize > maxsize) { stop("start model should not be of greater size than the max model size\n") } #Some error checking on the startmodel if(startmodelsize != 1) { for(i in 1:(startmodelsize-1)) { if(startmodel[i,1] ==0) { stop("first column of startmodel cannot be zero\n") } if(startmodel[i,2] == 1) { if(startknots[(i*2)-1] < min(predictors[,startmodel[i,1]]) || startknots[(i*2)-1] > max(predictors[,startmodel[i,1]])) { stop("Knot out of range of its predictor \n") } } if(startmodel[i,4] == 1) { if(startknots[(i*2)] <= min(predictors[,startmodel[i,3]]) || startknots[(i*2)] >= max(predictors[,startmodel[i,3]])) { stop("Knot out of range of its predictor\n") } } } if(max(startmodel[,c(1,3)]>npredictors)) { stop("Initial model misspecified on input\n") } } startmodel<-t(startmodel) resultmodelsize<-0 end.state<-0 step.count <-0 z <- .C("polymars", as.integer(npredictors), as.integer(nresponses), as.integer(ncases), as.double(datamatrix), as.integer(knots), as.double(mesh.vector), as.integer(mesh.specified), as.integer(maxsize), as.double(gcv), as.integer(additive), as.integer(startmodelsize), start.model=as.integer(startmodel), start.knots=as.double(startknots), as.integer(weighted), as.double(weights), as.integer(no.interact.size), as.integer(no.interact), as.integer(no.remove.size), as.integer(no.remove), as.integer(knot.space), as.integer(testset), as.double(testsetmatrix), as.integer(testsetcases), as.integer(testset.weighted), as.double(ts.weights), as.integer(classify), as.double(tolerance), as.integer(verbose), best.model = as.integer(matrix(nrow=maxsize, ncol=4, data=rep(0,maxsize*4))), coefficients = as.double(matrix(nrow=maxsize, ncol=nresponses, data=rep(0.0,maxsize*nresponses))), steps = as.integer(matrix(nrow=maxsize*2, ncol=2, data=rep(0,maxsize*4))), rss.gcv = as.double(matrix(nrow=maxsize*2, ncol=nresponses+1, data=rep(0.0,maxsize*2*(nresponses+1)))), modelsize=as.integer(resultmodelsize), modelknots = as.double(matrix(nrow=maxsize, ncol=2, data=rep(0.0,maxsize*2))), coefficient.se.term = as.double(rep(0.0,maxsize)), end.state = as.integer(end.state), step.count = as.integer(step.count)) #The C function returns information about how it ended if(z$end.state != 0 && z$end.state !=5) { switch(z$end.state, stop("Mis-specification of initial model\n"), stop("Initial model with non-linear function must contain the corresponding linear function\n"), stop("Initial model contains two-predictor functions that require prerequisite functions\n")) } else { model<-matrix(z$best.model[1:((z$modelsize-1)*4)],ncol=4,byrow=T) knot.values<-matrix(z$modelknots[1:((z$modelsize-1)*2)],ncol=2,byrow=T) for(i in 1:nrow(model)) { if(model[i,2] != 0){model[i,2]<-knot.values[i,1]}else{model[i,2]<-NA} if(model[i,4] != 0){model[i,4]<-knot.values[i,2]}else{model[i,4]<-NA} } if(length(knots[model[,1]]) !=0 && min(knots[model[,1]])<0) { factor1<-T levels1<-rep(NA,z$modelsize-1) factor.variables<-unique(model[knots[model[,1]]<0,1]) for( i in 1:length(factor.variables)) { for( j in 1:length(model[,1])) { if(model[j,1] == factor.variables[i]){levels1[j]<-model[j,2];} } model[model[,1] == factor.variables[i],2]<-NA } levels1<-c(NA,levels1) } else {factor1<-F} coefs<-matrix(z$coefficients[1:(z$modelsize*nresponses)],ncol=nresponses) #The model that the C-function returns does not explicitly contain an intercept #so in formatting the output one is added if(z$modelsize > 1) { if(factor1==F) { model<-rbind(c(0,NA,0,NA),model) model<-data.frame(model,coefs) if(nresponses == 1) { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","pred2", "knot2","coefs")) } else { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","pred2","knot2", paste("Coefs",1:nresponses))) } } if(factor1==T) { model[(knots[model[,1]]<0),2]<-NA model<-rbind(c(0,NA,0,NA),model) model<-data.frame(model[,1:2],levels1,model[,3:4],coefs) if(nresponses == 1) { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","level1", "pred2","knot2","coefs")) } else { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","level1", "pred2","knot2",paste("Coefs",1:nresponses))) } } } else { model<-data.frame(0,NA,0,NA,coefs) if(nresponses == 1) { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","pred2", "knot2","coefs")) } else { dimnames(model)<-list(1:z$modelsize,c("pred1","knot1","pred2","knot2", paste("Coefs",1:nresponses))) } } #for later plotting the ranges and medians of the predictors are stored ranges.and.medians<-matrix(ncol=npredictors,nrow=3,data=0) for(i in 1:npredictors) {ranges.and.medians[1,i]<-min(predictors[,i])} for(i in 1:npredictors) {ranges.and.medians[2,i]<-max(predictors[,i])} for(i in 1:npredictors) {ranges.and.medians[3,i]<-median(predictors[,i])} # A table with information from the fitting is formatted here steps<-matrix(z$steps[1:(2*(z$step.count+1))],ncol=2,byrow=T) rss.gcv<-matrix(z$rss.gcv[1:((nresponses+1)*(z$step.count+1))], ncol=nresponses+1, byrow=T) fitting<-data.frame(steps,rss.gcv) if(testset == F) { if(nresponses == 1) { dimnames(fitting) <- list(1:(nrow(fitting)), c("0/1","size","RSS","GCV")) } else { dimnames(fitting) <- list(1:nrow(fitting), c("0/1", "size", paste("RSS", 1:nresponses), "GCV")) } } else { if(classify ==F) { if(nresponses == 1) { dimnames(fitting) <- list(1:(nrow(fitting)), c("0/1","size","RSS","T.S. RSS")) } else { dimnames(fitting) <- list(1:nrow(fitting), c("0/1", "size", paste("RSS", 1:nresponses), "T.S. RSS")) } } else { if(nresponses == 1) { dimnames(fitting) <- list(1:(nrow(fitting)), c("0/1","size","RSS","T.S.M.C.")) } else { dimnames(fitting) <- list(1:nrow(fitting), c("0/1", "size", paste("RSS", 1:nresponses), "T.S.M.C.")) } } } #calculates fitted values and residual of the data according to the #model returned if(z$modelsize >1) { temp<-list(model=model, model.size = z$modelsize, ranges.and.medians=ranges.and.medians, responses = nresponses ) class(temp)<-"polymars" fitted<-matrix(ncol=nresponses, nrow=ncases, data=rep(0,nresponses*ncases)) residuals<-matrix(ncol=nresponses, nrow=ncases, data=rep(0,nresponses*ncases)) fitted<-predict.polymars(temp,x=predictors) residuals<-responses-fitted #model<-model[,-c(nrow(model)-nreponses,nrow(model))] } else { fitted<-matrix(ncol=nresponses,nrow=ncases,data=coefs[1,1]) residuals<-matrix(ncol=nresponses,nrow=ncases,data=responses-coefs[1,1]) } # if their are factors present in the model the factors must be stored for use during plotting if(factor1 == T) { model2<-model[-1,] factors.in.model<-unique(model2[knots[model2[,1]]<0,1]) maxfactors<-0 for(i in 1:length(factors.in.model)) { maxfactors<-max(maxfactors,length(unique(predictors[,factors.in.model[i]]))) } factor.matrix<-matrix(ncol=length(factors.in.model), nrow=maxfactors+2,data=NA) for(i in 1:length(factors.in.model)) { factor.matrix[1,i]<-factors.in.model[i] factor.matrix[2,i]<-length(unique(predictors[,factors.in.model[i]])) for(j in 3:(length(unique(predictors[,factors.in.model[i]]))+2)) { factor.matrix[j,i]<-unique(predictors[,factors.in.model[i]])[j-2] } } } else { factor.matrix<-0 } if(nresponses ==1) { SE <- round(sqrt((sum(residuals^2)/(ncases-z$modelsize))*z$coefficient.se.term[1:z$modelsize]),4) model<-cbind(model,SE) dimnames(model)[[2]][length(dimnames(model)[[2]])]<-"SE" } else { for(i in 1:nresponses) { SE <- round(sqrt((sum(residuals[,i]^2)/(ncases-z$modelsize))*z$coefficient.se.term[1:z$modelsize]),4) model<-cbind(model,SE) dimnames(model)[[2]][length(dimnames(model)[[2]])]<-paste("SE",i) } } if(nresponses ==1) { Rsquared<-1-(sum(residuals^2)/sum((responses-mean(responses))^2)) } else { Rsquared<-NULL } result<-list(model=model, fitting=fitting, model.size = z$modelsize, fitted=fitted, responses=nresponses, residuals=residuals, ranges.and.medians=ranges.and.medians, call=call, conversion=conversion, factor.matrix=factor.matrix, Rsquared=Rsquared) class(result)<-"polymars" return(result) } } ################################################################################################ predict.polymars<-function(mars.model,x,classify=F,intercept) { # Produces predicted values for a polymars object # mars.model an object returned from a call to polymars # x a matrix with number of columns equal to number of columns of predictor matrix in # original call to polymars and predictor values in the corresponding columns. Can # also be a matrix with number of column equal to the number of predictors in the # model, in the order of the original dataset. # classify If the original call to polymars was for classification setting classify=T will # the new data otherwise it will return the multi-response fitted values. # intercept By default T. The full intercept is included; or when F the intercept is left out. # Can also be givebn a numerical value if(missing(intercept)) { intercept<-T } # some error checking if(!(is.polymars(mars.model))) { stop("The first arguement must be an object returned from a call to `mars'\n") } # The x matrix number of columns can be of length equal to the number of # predictors in the original model or shorten to the number of predictors in # the model in `mars.model' if(!(is.matrix(x))) { if(length(unique(mars.model$model[, "pred1"]))== 1 || ncol(mars.model$ranges.and.medians)== 1 ) { x<-matrix(data=x,ncol=1) } } if((is.matrix(x) && ncol(x) != length(unique(mars.model$model[,"pred1"])))) { if(ncol(x) != ncol(mars.model$ranges.and.medians)) { stop("Input should be a matrix with number of columns equal to either number of original predictors or number of predictors in model\n") } } # If the number of columns of the matrix is not length equal to number of # predictors it is expanded to that size. if(is.matrix(x) && ncol(x) == length(unique(mars.model$model[, "pred1"])) && ncol(x) != ncol(mars.model$ranges.and.medians)) { tempmatrix<-x x<-matrix(nrow=nrow(tempmatrix),ncol=ncol(mars.model$ranges.and.medians),data = 0) for(i in 1:length(unique(mars.model$model[, "pred1"]))) { for(j in 1:nrow(tempmatrix)) { x[j,sort(unique(mars.model$model[,"pred1"]))[i]]<-x[j] } } } # If x is a vector put it into matrix form expanding it if it is of # length equal to only the number of predictors in the model in `mars.model' if(!(is.matrix(x))) { if(!(length(x) == ncol(mars.model$ranges.and.medians) || length(x) == unique(mars.model$model[, "pred1"]))) { stop("The vector of values must be equal in length to either the number of original predictors or predictors in the model\n") } if(length(x) == unique(mars.model$model[, "pred1"]) && length(x) != ncol(mars.model$ranges.and.medians)) { x <- rep(0, ncol(mars.model$ranges.and.medians)) for(i in 1:length(unique(mars.model$model[, "pred1"]))) { x[sort(unique(mars.model$model[, "pred1"]))[i]]<-x[i] } } x <- t(as.matrix(x)) } # Checking to see if there are factor variables in the model if(dimnames(mars.model$model)[[2]][3] == "level1") { level1<-T mars.model$model<-mars.model$model[,c(1:(5+mars.model$responses))] #if(dimnames(mars.model$model)[[2]][6] == "level2"){level2<-T}else{level2<-F} } else { level1<-F mars.model$model<-mars.model$model[,c(1:(4+mars.model$responses))] #if(dimnames(mars.model$model)[[2]][5] == "level2") # {level2<-T}else{level2<-F} } # Setting up the fitted responses matrix responses<-mars.model$responses Y <- matrix(ncol = responses, nrow = nrow(x), data = rep(0, nrow(x))) Y1 <- matrix(ncol = 1, nrow = nrow(x), data = rep(0, nrow(x))) Y2 <- matrix(ncol = 1, nrow = nrow(x), data = rep(0, nrow(x))) if(is.logical(intercept)) { if(intercept==T) { for(i in 1:responses)Y[,i] <- mars.model$model[1,ncol(mars.model$model)-responses+i] } else { if(intercept==F) { for(i in 1:responses)Y[,i] <- 0.0 } } } else { if(is.numeric(intercept)) { if(length(intercept)==responses) { for(i in 1:responses)Y[,i] <- intercept[i] } else { if(length(intercept) != 1) { stop("Intercept arguement mispecified \n") } for(i in 1:responses)Y[,i] <- intercept } } } # Computing fitted values if(mars.model$model.size>1) { for(i in 2:mars.model$model.size) { Y2[] <- 1 Y1[] <- x[,mars.model$model[i, "pred1"]] if(!is.na(mars.model$model[i, "knot1"])) { Y1 <- Y1 - mars.model$model[i,"knot1"] Y1[Y1 < 0,] <- 0 } if(level1) { if(!is.na(mars.model$model[i, "level1"])) { Y1<- (Y1 == mars.model$model[i, "level1"]) } } if(!is.na(mars.model$model[i, "pred2"]) & mars.model$model[i, "pred2"] != 0) { Y2[] <- x[,mars.model$model[i,"pred2"]] if(!is.na(mars.model$model[i,"knot2" ])) { Y2 <- Y2 - mars.model$model[i,"knot2"] Y2[Y2 < 0,] <- 0 } #if(level2) #{ # if(!is.na(mars.model$model[i, "level2"])) # { # Y2<- (Y2 == mars.model$model[i, "level2"]) # } #} } for(j in 1:responses){Y[,j]<-Y[,j]+(Y1 * Y2 * mars.model$model[i,ncol(mars.model$model)-responses+j])} } } # If classification is to be used the original polymars fitting expanded the # response into a vector of indicator variables. The largest of the responses # correspondes to the fitted class for each case. if(classify == T) { for(i in 1:nrow(Y)) { Y[i,]<-Y[i,]==max(Y[i,]) } if(is.matrix(mars.model$conversion)) Z<-Y Y<-matrix(ncol=1,nrow=nrow(Z)) for(i in 1:nrow(Y)) { for(j in 1:ncol(Z)) { if(Z[i,j] == 1) Y[i,] <- mars.model$conversion[j] } } } else { # if classification was used but the full multiple response fitted response # matrix is requested the response names (corresponding to the classes) are # added. if(is.matrix(mars.model$conversion)) { dimnames(Y)[[2]]<-list(mars.model$conversion[,1]) } } return(Y) } ################################################################################################ is.polymars<-function(object) { if(is.na(class(object)[1]=="polymars")){return(F)} if(class(object) == "polymars") {return(T)} else{return(F)} } ################################################################################################ print.polymars<-function(mars.model) { UseMethod("summary") } ################################################################################################ summary.polymars<-function(mars.model) { cat("Call:\n") print(mars.model$call) cat("\nModel fitting\n\n") print(mars.model$fitting) cat("\n\nModel produced\n\n") print(mars.model$model) if(mars.model$responses != 1) cat("\nRESPONSES :", mars.model$responses, "\n") if(!is.null(mars.model$Rsquared)) cat("\nRsquared :",round(mars.model$Rsquared,3),"\n") invisible() } plot.polymars<-function(mars.model,predictor1,response,predictor2,x,add=F,n,xyz=F,contour.ploymars=F,xlim,ylim,main,intercept,...) { # mars.model a polymars object # predictor1 the column number in the original predictor matrix of the predictor of interest # response with multi-response polymars the column number in the original response matrix. # Default is 1 # predictor2 the second predictor for a contour or persp plot. For single response data # plot(pmars1,2,6) is understood as 3-d plot of predictors 2 and 6. # x Values for the other predictors can be given, using the same format as for the # predict fuhnction. By default median values are used. # add should the plot be added to another. (for 2-d plots only) # n For 2-d plot the number of points the function is interploted over. For 3-d plots # the a n*n mesh is interploted over. Default 2-d: 100, 3-d 33. # xyz sometimes a call can be ambiguous: plot(pmars1,6,2) a 2-d plot with 2nd response # or a 3-d plot. Use xyz=T for 3-d. #contour.ploymars By default a 3-d if a `persp' plot. contour.ploymars=T asks for a `contour' plot. #intercept same as for predict function. =T intercepr is included =F it is left out, or can # be given a numerical value. if(missing(x))x<-mars.model$ranges.and.medians[3,] if(length(x) != ncol(mars.model$ranges.and.medians)) { stop("x should be of length equal to the number of predictors in original data\n") } if(missing(predictor2) && (!missing(response)) && mars.model$responses == 1) { if(missing(predictor1) && xyz == T) { stop("You must specify 2 predictor numbers") } xyz<-T predictor2<-response response<-1 } if(contour.ploymars == T) { xyz<-T } if(missing(intercept)) { intercept<-T } if(xyz==T) { if(missing(n))n<-33 if(missing(response)) { if(missing(xlim)) { polymars.persp(mars.model, predictor1, predictor2, n=n, contour.ploymars=contour.ploymars, intercept=intercept, ...) } else { polymars.persp(mars.model, predictor1, predictor2, n=n, xlim=xlim, contour.ploymars=contour.ploymars, intercept=intercept, ...) } } else { if(missing(xlim)) { polymars.persp(mars.model, predictor1, predictor2, response, n=n, contour.ploymars=contour.ploymars, intercept=intercept, ...) } else { polymars.persp(mars.model, predictor1, predictor2, response, n=n, xlim=xlim, contour.ploymars=contour.ploymars, intercept=intercept, ...) } } invisible(return()) } else { if(missing(predictor1)) { cat("predictor should be specified \n") } if(mars.model$responses != 1 && missing(response)&& missing(predictor2)) { cat("Response should be specified (default: response =1)\n") } #check to see that the predictor is in the model inmodel<-F for(i in 2:mars.model$model.size) { if(mars.model$model[i,"pred1"] == predictor1)inmodel<-T } if(inmodel == F) { stop("The predictor specified is not in the model\n") } #check to see if the predictor is a factor if(is.matrix(mars.model$factor.matrix)) { if(length(mars.model$factor.matrix[1,mars.model$factor.matrix[1,]== predictor1]) != 0) { isfactor<-T } else { isfactor<-F } } else { isfactor<-F } if(isfactor == T) { pred.values <- matrix(nrow = mars.model$factor.matrix[2,mars.model$factor.matrix[1,]==predictor1], ncol = ncol(mars.model$ranges.and.medians),data = x, byrow = T) factors<-mars.model$factor.matrix[-c(1,2),mars.model$factor.matrix[1,]==predictor1] pred.values[,predictor1]<- factors[!is.na(factors)] mesh<-factors[!is.na(factors)] } else { if(missing(n))n<-100 if(missing(xlim))xlim<-c(mars.model$ranges.and.medians[1,predictor1],mars.model$ranges.and.medians[2,predictor1]) pred.values <- matrix(nrow = n, ncol = ncol(mars.model$ranges.and.medians), data = x, byrow = T) mesh <- matrix(seq(xlim[1],xlim[2],(xlim[2]-xlim[1])/(n-1)),nrow=1) pred.values[,predictor1]<-mesh } if(missing(response) && missing(predictor2))response<-1 if(response > mars.model$responses || response < 0) { stop("response arguement = ",response,"is out of range\n") } model<-mars.model$model Y<-predict.polymars(mars.model,pred.values,intercept=intercept) if(isfactor == F) { if(add == F) { if(mars.model$responses == 1) { plot(mesh,Y,...,type="l",xlab=paste("Predictor ",predictor1),ylab="Response") } else { plot(mesh, Y[,response], type="l", xlab=paste("Predictor ",predictor1), ylab=paste("Response ",response), ...) } } else { points(mesh, Y, type="l") } } if(isfactor == T) { if(add == F) { if(mars.model$responses == 1) { plot(mesh,Y,...,xlab=paste("Predictor ",predictor1),ylab="Response") } else { plot(mesh, Y[,response], type="l", xlab=paste("Predictor ",predictor1), ylab=paste("Response ",response), ...) } } else { points(mesh, Y, type="l") } } invisible() } } ################################################################################################ polymars.persp<-function(mars.model, predictor1, predictor2, response, n= 33,xlim,ylim,x,contour.ploymars,main,intercept,...) { # used by the plot.polymars function # not designed for stand alone use. if(missing(x))x<-mars.model$ranges.and.medians[3,] if(missing(xlim))xlim<-c(mars.model$ranges.and.medians[1,predictor1],mars.model$ranges.and.medians[2,predictor1]) if(missing(ylim))ylim<-c(mars.model$ranges.and.medians[1,predictor2],mars.model$ranges.and.medians[2,predictor2]) if(missing(predictor1) || missing(predictor2)) { stop("You must specify 2 predictor numbers\n") } if(mars.model$responses != 1 && missing(response)) { cat("Response should be specified (default: response =1)\n") } if(missing(response))response <- 1 if(response > mars.model$responses || response < 0) { stop("response arguement = ",response,"is out of range\n") } if(sum(as.integer(predictor1==mars.model$model[,1])) == 0) { stop("Predictor 1 not in model\n") } if(sum(as.integer(predictor2==mars.model$model[,1])) == 0) { stop("Predictor 2 not in model\n") } X <- seq(xlim[1],xlim[2],(xlim[2] - xlim[1])/(n-1)) y <- seq(ylim[1],ylim[2],(ylim[2] - ylim[1])/(n-1)) meshX <- rep(X, n) meshY <- rep(y, n) meshY <- sort(meshY) pred.values <- matrix(nrow = n^2, ncol = ncol(mars.model$ranges.and.medians), data = x, byrow = T) for(i in 1:(n^2))pred.values[i, predictor1] <- meshX[i] for(i in 1:(n^2))pred.values[i, predictor2] <- meshY[i] Z <- predict.polymars(mars.model, pred.values,intercept=intercept)[,response] Z <- matrix(Z, ncol = n, byrow = F) xtitle<-paste("Predictor", predictor1) ytitle<-paste("Predictor", predictor2) if(mars.model$responses > 1) { if(missing(main) && (!contour.ploymars)) { ztitle <- paste("Response", response) } if(missing(main) && (contour.ploymars)) { ztitle <- paste("Contour of response",response) } } else { if(missing(main) && (!contour.ploymars))ztitle <- "Response" if(missing(main) && contour.ploymars)ztitle <- paste("Contour of response") } if(!contour.ploymars) { persp(X, y, Z, xlab = xtitle, ylab= ytitle, zlab = ztitle) } else { contour(X, y, Z, xlab = xtitle, ylab = ytitle , main = ztitle) } invisible() } ################################################################################################ SHAR_EOF fi if test -f 'polymarsall.c' then echo shar: "will not over-write existing file 'polymarsall.c'" else cat << \SHAR_EOF > 'polymarsall.c' /* Time-stamp: <97/07/09 12:56:43 martin> */ /****************************************************************************** * (c) Charles Kooperberg and Martin O'Connor 1997 * * This function is part of an implementation of the Multivariate Adaptive * * Regression Splines (MARS) methodology, first proposed by J.H. Friedman(1991)* * The Annals of Statistics, 19, 1 - 141. * * The program is a modified version of MARS and is described in * * Kooperberg, C., Bose, S. and Stone C.J.(1997) ``Polychotomous Regression'', * * 92, 117 - 127. * * Journal of the American Statistical Association * * You are free to use this program, for non-commercial purposes only, * * under the condition that * * this note is not to be removed. * * * * The program is not formally maintained, but we are interested in hearing * * from people who have problems with it, although we may not be able to solve * * them. * * Email clk@fhcrc.org or martin@stat.washington.edu * * Charles Kooperberg and Martin O'Connor, May 20, 1997 * ******************************************************************************* /* This file contains the main body of the program to be loaded into Splus. */ /* The only other file needed is ``f2c.h'' */ #include #include #include "f2c.h" #define calloc S_alloc #define TRUE 1 #define FALSE 0 double tolerance; int predictors;/*number of predictors in the dataset*/ int responses;/*number of responses in dataset*/ int cases;/*number of cases in dataset*/ int max_knots;/*maximum number of knots allowed in any of the predictors*/ /*This is used for memory allocation */ int model_size; /*holds current model size in the fitting procedure*/ int *knots_per_pred;/*a pointer to an array which holds the number of knots */ /*availible for each predictor in the model fitting */ int max_model_size;/*maximum size that the model is allowed to grow to in */ /*fitting procedure*/ int *order_keeper1;/*These matricies hold information about the oder in which*/ int *order_keeper2;/*the spline functions can be added. A predictor must*/ int *order_keeper3;/* have a linear spline before one with a knot*/ int *best_model;/*A pointer to an array that holds the best model*/ int additive;/*A boolean as to whether the user wanrts an additive model or not.*/ double GCV;/*A boolean as to whether the user wanrts an additive model or not.*/ /*measure for the best model so far. Used as critirium for choosing */ /*best overall model.*/ double GCVconstant;/*This value is used in calculating GCV and can be specified by */ /*the user.*/ double *best_coefficents;/*This is a pointer to an array that stores the coefficients*/ /*of the best model so far*/ double *coefficents;/*Used in testset to calculate coefficients for testset RSS. */ /*Usually the coefficeients are stored in a standardised form*/ /*for numerical stability.*/ int *steps, step_count ;/*Information about each step of the fitting is recorded*/ int *bestmodel_size;/*A pointer to the size of best bestmodel. The variable is */ /*passed in through Splus.*/ double *best_XtXinv;/*The X transpose X matrix of the best model */ /* Used to compute the standard errors of the coefficients*/ double *coef_sd_const;/*Constants to multiply the residual standard error by */ /* to get SEs for coefficients */ double *rssgcv;/*A pointer to an array passed from Splus. Details of RSS and GCV */ /*for each step stored here.*/ int *startmodel;/*A starting model may be passed in by Splus*/ double *startknots;/*A pointer to an array passed from Splus number of knots for */ /* specified by user*/ int *weighted;/*A boolean as to whether weights are specified for the model fitting*/ double weight_sum;/*A sum of the weights- used in calculating GCV in weighted */ int interaction_specs_size;/*If certain predictor are not to has interaction*/ int *interaction_specs; /* terms together they are specified in an array*/ /*with the size of this array (divided by 2).*/ int not_remove_size; /*If certain terms in the start model are not to be removed*/ int *not_remove_specs;/* an array holds the information and a variable hold the */ /* size of this array*/ int knot_space;/* The minimum number of order statistics between knots allowed when */ /* the possible knots are computed(compute_mesh)*/ int testset;/*A boolean as to whether the model are selected using a testset or not. */ double *testset_weights;/*A pointer to an array of testset weights*/ int testset_weighted;/*A boolean as to whether weights are specified for testset*/ double *tset_RSS;/*Used to hold testset RSS in function testset_RSS*/ int *response_class;/*The class assigned to a fitted response in testset_RSS when*/ /*using classification.*/ double *response_max;/* Used to classify in testset_RSS when using classification.*/ int classification;/*A boolean to use classification or not(normally false)*/ int Verbose;/*A boolean to use have the function printout as it goes along*/ struct matrix1 *weight_matrix; struct matrix1 *YtY;/* Y are the responses-this is Y-transpose by Y*/ struct matrix1 *data_matrix; struct matrix1 *testset_matrix;/* testset datamatrix*/ struct matrix1 *X_matrix;/* X is the design matrix*/ struct matrix1 *XtX_inverse; struct matrix1 *XtX_newinverse; struct matrix1 *new_XtXcolumn; struct matrix1 *function_values;/*- list of function values at knot location-*/ struct matrix1 *function_values_2;/*used in computing inner products*/ struct matrix1 *Rao_B;/*These are matricies used in computing a new XtX*/ struct matrix1 *Rao_F;/*matrix from an X matrix that has 1 column more or less*/ struct matrix1 *Rao_E;/*than the previous one. From an example in Rao's linear*/ struct matrix1 *Rao_E_inverse;/* algebra */ struct matrix1 *Rao_F_E_inverse_Ft; struct matrix1 *Rao_F_E_inverse; struct matrix1 *YtXXtX_newinverseXtY; struct matrix1 *XtX_newinverseXtY; struct matrix1 *YtX; struct matrix1 *new_X_matrix; struct matrix1 *temp_matrix;/*Used in matrix multiplication when weights are*/ /* multiplied in fit_as_candidate and initial_model*/ char *S_alloc(); double *best_model_sd_mean; /*all columns of the X matrix are standardised*/ /* coefficients of the best model */ double *model_sd_mean; /*----------------------------------------------------------------*/ struct link{ struct link *next; double *data; struct basis_function *function; }; struct list { struct link *list; int length; }; struct matrix1 { double *matrix; int nrow; int ncol; }; /*--this matrix is used for the YtransposeY and XtY matricies as column must be added and switched around--*/ struct matrix2 { struct link *column_list; int nrow; int ncol; }; struct basis_function /* A basis function is a spline (truncated power spline) or tensor product of a spline. These may have knots(elbow functions). There entries in the design (X) matrix are given as z-scores : minus the mean of the column then diveded by the standard dev. */ { int predictor1; int knot1_index; double knot1_value; int predictor2; int knot2_index; double knot2_value; struct basis_function *link; double SD; double mean; }; struct basis_function_matrix /* This is a matrix with number of rows equal to the number of predictors and number of columns variable for each row depending on the number of basis functions that are either candidates or in the model for that predictor (there is a model matrix and a candidates matrix. */ { struct basis_function *functions; int predictor_index; int number_of_basis_functions; struct basis_function_matrix *next_predictor; }; /*Two types of matrix used */ struct matrix1 *create_matrix1(int nrow, int ncol); struct matrix2 *create_matrix2(int nrow, int ncol); /* for matrix2*/ void switch_columns(int col1, int col2, struct matrix2 *object_matrix); /*void print_matrix1(struct matrix1 *object_matrix );*/ /*void print_matrix2(struct matrix2 *object_matrix );*/ /*to multiply matricies of type 1*/ void matrix_multiplication1(struct matrix1 *object_matrixA, struct matrix1 *object_matrixB, struct matrix1 *result, int flag); /*to multiple matrix of type 1 to type 2 */ void matrix_multiplication2(struct matrix1 *object_matrixA, struct matrix2 *object_matrixB, struct matrix1 *result, int flag); /* prints a basis_function_matrix */ /*void print_functions(struct basis_function_matrix *functions_matrix);*/ /*------functions --*/ /* find possible knot locations for basis functions. Subset of order statistics. An array of double. knots_per_pred is used to access this array. Also holds levels of categorical variables---*/ double *compute_mesh(); /* Sets up dynamically created variables controlls model selection*/ int fit_model(double *mesh); /*find candidate basis functions*/ void find_candidates(struct matrix2 *YtXXtX_expanded, double *mesh, struct basis_function_matrix *model, struct basis_function_matrix *candidates); /*Is a function already in the model*/ int in_model(int predictor1, int knot1_index, int predictor2, int knot2_index, struct basis_function_matrix *model); /*Compute the relevant inner products for YtX and XtX as if it was in the model*/ void fit_as_candidate(int predictor1, int knot1_index, int predictor2, int knot2_index, struct matrix2 *YtXXtX_expanded, double *mesh); /*Function checks whether a basis function can become a new candidate and if it can put it in the candidates matrix*/ int new_candidate(int predictor1, int knot1_index, int predictor2, int knot2_index, struct basis_function_matrix *candidates, struct basis_function_matrix *model, struct matrix2 *YtXXtX_expanded); /*find the candidate to be added to the model*/ int find_best_candidate(struct matrix2 *YtXXtX_expanded, struct basis_function_matrix *model, struct basis_function_matrix *candidates, double *mesh); /*After the best candidate to be added is found the modelis updated*/ void update_model(struct matrix2 *YtXXtX_expanded, struct basis_function_matrix *model, struct basis_function_matrix *candidates, int candidate, double *mesh); /*the deletion stage consists of one function to decide which basis function to delete and updates everything*/ int reduce_model(struct matrix2 *YtXXtX_expanded, struct basis_function_matrix *model); /* fits an initial model if one is specified*/ int initial_model(struct basis_function_matrix *model, struct matrix2 **YtXXtX_expanded); /* standardises each column of the X matrix substracting the mean and dividing by the standard deviation*/ void standardise_array(double *numbers,int length_of_list,double *mean,double *SD); /*checks the initial model if given, as it must be consistant with how the proceure adds functions*/ int check_input(); /*use lapack inversion function. Some preprocessing of the matrix in done first*/ int invert_matrix(struct matrix1 *object_matrix); /* computes testset RSS if required*/ double testset_RSS(struct matrix2 *YtXXtX_expanded,int model_size); /*----------------------------------------------------------------*/ /* This function is called from Splus function polymars*/ /*==============================================================*/ void polymars(int *pred, int *resp, int *ncases, double *datamatrix, int *knotinfo, double *given_mesh, int *is_mesh_specified, int *maxmodel, double *gcvvalue, int *addflag, int *start_model_size, int *start_model, double *start_knots, int *weights_indictor, double *caseweights, int *nointeraction, int *nointeractionrule, int *noremove, int *noremoverule, int *knotspace, int *testset_flag, double *testsetmatrix, int *testset_ncases, int *testset_weights_indictor, double *testset_weights, int *classify, double *stability1, int* verbose, int *model_returned, double *coefs_returned, int *steps_in_fitting, double *rss_and_gcvs, int *resultmodel_size, double *model_knots, double *sd_constants, int *end_condition, int *nstep) /*==============================================================*/ { double *mesh,*mesh_ptr;/*mesh is the matrix of possible knot values*/ int i,j,k; /*Matching up the S variables with global variables in this file */ startmodel = start_model; startknots = start_knots; model_size = *start_model_size; weighted = weights_indictor; step_count = 0; predictors = *pred; responses = *resp; cases = *ncases; GCVconstant = *gcvvalue; max_model_size = *maxmodel; knots_per_pred = knotinfo; additive = *addflag; bestmodel_size = resultmodel_size; *bestmodel_size = 1; interaction_specs_size = *nointeraction; interaction_specs = nointeractionrule; not_remove_size = *noremove; not_remove_specs = noremoverule; knot_space = *knotspace; testset = *testset_flag; classification = *classify; testset_weighted = *testset_weights_indictor; best_model = model_returned; best_coefficents = coefs_returned; coef_sd_const = sd_constants; steps = steps_in_fitting; rssgcv = rss_and_gcvs; tolerance = *stability1; Verbose = *verbose; data_matrix = (struct matrix1 *)calloc(1,sizeof(struct matrix1)); data_matrix->ncol = predictors + responses; data_matrix->nrow = cases; data_matrix->matrix = datamatrix; if(*testset_flag == TRUE) { testset_matrix = (struct matrix1 *)calloc(1,sizeof(struct matrix1)); testset_matrix->ncol = predictors + responses; testset_matrix->nrow = *testset_ncases; testset_matrix->matrix = testsetmatrix; } if(*weighted == TRUE) { weight_sum = 0.0; for(i=0;inrow = cases; weight_matrix->ncol = cases; weight_matrix->matrix = caseweights; } else { weight_sum = cases; } *end_condition =0;/* records why the program ended*/ /*checking that the input is consistant with requirements of the procedure*/ /*Just checks that a startmodel doesn't contradict the hierarchy rules of which*/ /*terms must be in the model before which*/ *end_condition = check_input(); if(*end_condition ==0) { if(*is_mesh_specified == FALSE) {mesh = compute_mesh();} else {mesh = given_mesh;} *end_condition = fit_model(mesh); /* number of steps of adding deleting given to variable to return to Splus*/ *nstep = step_count; } if(*bestmodel_size !=1) { /*get the knots for the model from the mesh to return to the Splus function. Only its index has been recorded by which it is found in the 'mesh'*/ for(i=0;i<2*(*bestmodel_size-1);i++) { mesh_ptr = mesh; if(best_model[(2*i)+1]!=0) { for(j=0;jmatrix; mesh_size = 0; /*----------------------------------- Knots in the initial model are treated as extra and put in after the usual knots for each predictor -------------------------------------*/ if(model_size >1) { for(i=0;i<2*(model_size-1);i++) { if(startmodel[(i*2)+1] == 1 && knots_per_pred[startmodel[(i*2)]-1] >-1) { knots_per_pred[startmodel[(i*2)]-1]++; } } } /*------------------------------------------*/ /*--counting the total number of knots--*/ for(i=0;i=0) { mesh_size = mesh_size + knots_per_pred[i]; } } /*---levels is an array to hold the different levels of the categorical variables. The levels will be put into the mesh matrix as knots are put in for continuous variables it is also used to sort predictor values into order statistics to calculate knots in continuous predictors-*/ levels = (double *)calloc(cases,sizeof(double)); /*-the number of levels per pred are added to mesh size--*/ for(i=0;imatrix[(responses+i)*cases]; k=0; for(j=0;jmatrix[(responses+i)*cases]; k=0; for(j=0;jmatrix[(responses+i)*cases]; /*-add new level to levels matrix-*/ for(j=0;j=0 && levels[k] >data_value) { levels[k+1]=levels[k]; k--; } levels[k+1]=data_value; } /*m counts the number of values that are the present more than once in a predictor*/ m = 0; for(j=0;j=knot_space) && (j <=cases-m-knot_space) ) { *mesh_ptr = levels[j]; mesh_ptr++; l = j; } else { if(knots_per_pred[i]>0) { knots_per_pred[i]=knots_per_pred[i]-1; } } } knots = knots_per_pred[i]; k=1; for(j=0;j<(model_size-1)*2;j++) { if(startmodel[j*2]-1 == i && startmodel[(j*2)+1]==1) { *mesh_ptr =startknots[j]; startmodel[(j*2)+1] = knots+k-nstartknots; k=k+1; mesh_ptr++; } } } } } return mesh; } /*==============================================================*/ int fit_model(double *mesh) /*==============================================================*/ { /*-------------------------------------------------------------- YtXXtX_expanded is a matrix of YtX and XtX together, the first rows being the YtX part where Y is the matrix of responses and X is the design matrix for basis function. --------------------------------------------------------------*/ struct basis_function_matrix *predictor_basis_functions; struct basis_function_matrix *candidate_basis_functions; struct basis_function_matrix *model=0; struct basis_function_matrix *candidates=0; int i,j,k,ok; int col_minder; struct matrix2 *YtXXtX_expanded; double standardise_const; double intercept_sd; if(Verbose == TRUE) { printf("\n"); } /* gcv measure of fit to find overall model*/ GCV= -1.0; /*--allocating space for data structures matrix sizes are calculated as the maximum possible needed */ XtX_inverse = create_matrix1(0,0); XtX_inverse->matrix = (double *)calloc(max_model_size*max_model_size,sizeof(double)); XtX_newinverse = create_matrix1(0,0); XtX_newinverse->matrix = (double *)calloc(max_model_size*max_model_size,sizeof(double)); new_XtXcolumn = create_matrix1(0,0); new_XtXcolumn->matrix = (double *)calloc(max_model_size,sizeof(double)); function_values = create_matrix1(0,0); function_values->matrix = (double *)calloc(cases,sizeof(double)); function_values->nrow=cases; function_values->ncol = 1; function_values_2 = create_matrix1(0,0); function_values_2->matrix = (double *)calloc(cases,sizeof(double)); function_values_2->nrow= 1; function_values_2->ncol = 1; Rao_B = create_matrix1(0,0); Rao_B->matrix = (double *)calloc(max_model_size-1,sizeof(double)); Rao_F = create_matrix1(0,0); Rao_F->matrix = (double *)calloc(max_model_size-1,sizeof(double)); Rao_E = create_matrix1(0,0);/*always a 1x1 matrix*/ Rao_E_inverse = create_matrix1(0,0); Rao_F_E_inverse_Ft = create_matrix1(0,0); Rao_F_E_inverse_Ft->matrix = (double *)calloc((max_model_size-1)*(max_model_size-1),sizeof(double)); Rao_F_E_inverse = create_matrix1(0,0); Rao_F_E_inverse->matrix = (double *)calloc(max_model_size-1,sizeof(double)); YtXXtX_newinverseXtY = create_matrix1(0,0); YtXXtX_newinverseXtY->matrix = (double *)calloc(responses*responses,sizeof(double)); XtX_newinverseXtY = create_matrix1(0,0); XtX_newinverseXtY->matrix = (double *)calloc(max_model_size*responses,sizeof(double)); YtX = create_matrix1(0,0); YtX->matrix = (double *)calloc(max_model_size*responses,sizeof(double)); new_X_matrix = create_matrix1(0,0); new_X_matrix->matrix = (double *)calloc(max_model_size*cases,sizeof(double)); temp_matrix = create_matrix1(0,0); if(responses < max_model_size){i = max_model_size;}else{i=responses;} temp_matrix->matrix = (double *)calloc(i*cases,sizeof(double)); best_model_sd_mean = (double *)calloc(max_model_size*2,sizeof(double)); if(testset == TRUE) { model_sd_mean = (double *)calloc(max_model_size*2,sizeof(double)); coefficents = (double *)calloc(max_model_size*responses,sizeof(double)); if(classification == FALSE) { tset_RSS = (double *)calloc(responses,sizeof(double)); } else { response_class = (int *)calloc(cases,sizeof(int)); response_max = (double *)calloc(cases,sizeof(double)); } } best_XtXinv = (double *)calloc(max_model_size*max_model_size,sizeof(double)); /*---Creating structure to hold the model functions------------------*/ for(i = 0;inext_predictor = model; predictor_basis_functions->number_of_basis_functions=0; predictor_basis_functions->predictor_index=predictors-i; model = predictor_basis_functions; } /*---Creating the structure for candidate functions--------------*/ for(i = 0;inext_predictor = candidates; candidate_basis_functions->number_of_basis_functions=0; candidate_basis_functions->predictor_index=predictors-i; candidates = candidate_basis_functions; } /*--computes YtY ----Y is just the first column(s) of the data matrix---*/ col_minder = data_matrix -> ncol; data_matrix -> ncol = responses; /*--reduce the dimensions of the matrix to just the part contain Y for multiplication--*/ YtY=create_matrix1(0,0); YtY->nrow=responses; YtY->ncol=responses; YtY->matrix = (double *)calloc (responses*responses,sizeof(double)); matrix_multiplication1(data_matrix,data_matrix,YtY,1); data_matrix -> ncol =col_minder; /*--create matricies that control the order of addition and deletion. A non-linear basis function can be added only where a linear one already exists--*/ /*-- these matricies hold information on whether linear functions are in model as a function with a knot can only be added after a linear function ------------------------------------------------*/ order_keeper1 = (int *)calloc(predictors,sizeof(int)); for(i = 0;imax_knots) { max_knots = knots_per_pred[i]; } } } if(max_knots < 0)max_knots = 0; order_keeper2 = (int *)calloc(predictors*predictors,sizeof(int)); for(i = 0;inrow = model_size; temp_matrix->ncol = cases; matrix_multiplication1(X_matrix,weight_matrix,temp_matrix,3); matrix_multiplication1(temp_matrix,X_matrix,XtX_inverse,0); } else { matrix_multiplication1(X_matrix,X_matrix,XtX_inverse,1); } ok = invert_matrix(XtX_inverse); } } j=model_size; for(i=0;i 1) { coef_sd_const[0] = best_XtXinv[0]; for(i=1;i<*bestmodel_size;i++) { coef_sd_const[0] = coef_sd_const[0] + ((best_model_sd_mean[((i-1)*2)+1]*best_model_sd_mean[((i-1)*2)+1])/ (best_model_sd_mean[((i-1)*2)]*best_model_sd_mean[((i-1)*2)]))* best_XtXinv[i+(*bestmodel_size)*i]; } for(i=1;i<*bestmodel_size;i++) { coef_sd_const[0] = coef_sd_const[0] - 2*best_XtXinv[i]*best_model_sd_mean[((i-1)*2)+1]/ best_model_sd_mean[((i-1)*2)]; } for(i=1;i<*bestmodel_size;i++) { for(j=i+1;j<*bestmodel_size;j++) { coef_sd_const[0] = coef_sd_const[0] + 2*best_XtXinv[i*(*bestmodel_size)+j] *(best_model_sd_mean[((i-1)*2)+1]/ best_model_sd_mean[((i-1)*2)]) *best_model_sd_mean[((j-1)*2)+1]/ best_model_sd_mean[((j-1)*2)]; } } } else { coef_sd_const[0] = 1; } /*For the standard errors of the non-transformed basis functions the variances of the transformed functions are adjusted using their SD, means and covariances*/ for(i=1;i<*bestmodel_size;i++) { coef_sd_const[i]=best_XtXinv[i*(*bestmodel_size)+i] /(best_model_sd_mean[((i-1)*2)]*best_model_sd_mean[((i-1)*2)]); } for(i=0;i=0) { if(!(in_model(i+1,0,0,0,model)) && new_candidate(i+1,0,0,0,candidates,model,YtXXtX_expanded)) { fit_as_candidate(i+1,0,0,0,YtXXtX_expanded,mesh); } } /*if the linear function has been added basis functions with knots can be added to candidates. In any case if it is categorical all possible levels become candidates at start.*/ if(current_predictor->number_of_basis_functions !=0 || (knots_per_pred[i]<0 &&model_size==1) ) { for(j=0;j<(int)abs(knots_per_pred[i]);j++) { if(!(in_model(i+1,j+1,0,0,model))&& new_candidate(i+1,j+1,0,0,candidates,model,YtXXtX_expanded)) { fit_as_candidate(i+1,j+1,0,0,YtXXtX_expanded,mesh); } } } /*categorical variables are not allowed in interactions--*/ if(additive == FALSE || knots_per_pred[i] <0) { /*--------------------------------------------------------------------------- two term candidates - interaction terms every predictor function in the model can be combined with another according to hierarchical rules ----------------------------------------------------------------------------*/ if(current_predictor->number_of_basis_functions !=0 && inext_predictor; for(j=i+1;jnumber_of_basis_functions !=0) { function_in_model = in_model(i+1,0,j+1,0,model); /*never true for categorical variables--*/ if((!function_in_model) && new_candidate(i+1,0,j+1,0,candidates,model,YtXXtX_expanded)) { fit_as_candidate(i+1,0,j+1,0, YtXXtX_expanded, mesh); } if(function_in_model) { for(k=0;knext_predictor; } } } current_predictor = current_predictor->next_predictor; } return; } /*==============================================================*/ int in_model(int predictor1, int knot1_index, int predictor2, int knot2_index, struct basis_function_matrix *model) /*==============================================================*/ { /* Check to see if a certain function given by its knots and predictor numbers is in the model */ /* the basis function is specified by it's predictor an knot indicies*/ struct basis_function_matrix* current_predictor; struct basis_function *current_function; int i, predictor_functions; current_predictor = model; if(model_size == 1) { return FALSE; } else { while(current_predictor->predictor_index < predictor1) { current_predictor = current_predictor->next_predictor; } predictor_functions = current_predictor->number_of_basis_functions; if(predictor_functions==0) { return FALSE; } current_function = current_predictor->functions; /*scan through the predictors basis functions looking for a match*/ /*Predictor1 is always a lower value than predictor2 so there isn't a basis function that is symatric (with predictor1 and predictor2 swapped)*/ for(i=0;iknot1_index == knot1_index && current_function->predictor1==predictor1 && current_function->knot2_index == knot2_index && current_function->predictor2== predictor2) { return TRUE; } if(i != predictor_functions-1) { current_function= current_function->link; } } } return FALSE; } /*--------------------------------------------------------------*/ void fit_as_candidate(int predictor1, int knot1_index, int predictor2, int knot2_index, struct matrix2 *YtXXtX_expanded, double *mesh) /*--------------------------------------------------------------*/ { /* fits a candidate which was found in "find candidate". The column of YtXXtX and the entry in the candidates matrix are already in place. It must evaluate the inner product between the new candidate and the functions already in the model also its inner product with the responses and all is stored in the YtXXtX matrix*/ int i,j; double *current_predictor1,*current_predictor2; int mesh_index; double knot1_value,knot2_value; double entry; struct link *new_column;/*----index of last row of matrix-----*/ double *function_value,SD,mean; /*--function value stores the value of the function at the predictor values for each case */ /*---put new entries into YtXXtX for new candidate-------------------*/ new_column= YtXXtX_expanded->column_list; for(i=0;i ncol-1;i++) { new_column=new_column->next; } /*--evaluate and store candidate function value for each case ---*/ function_value = function_values->matrix; /*-find the knot values using the indicies for the mesh----------*/ if(knot1_index != 0) { mesh_index=0; for(i=0;ifunction->knot1_value = knot1_value; } if(knot2_index != 0) { mesh_index=0; for(i=0;ifunction->knot2_value = knot2_value; } /*--find the data values for the corresponding predictor(s) in the function move to row of predictor of interest in data matrix---*/ current_predictor1 = &data_matrix->matrix[((predictor1-1)+responses)*cases]; current_predictor2 = &data_matrix->matrix[((predictor2-1)+responses)*cases]; for(i=0;i=0) { *function_value = *current_predictor1; if(knot1_index!=0) { *function_value = *function_value - knot1_value; if(*function_value <0){*function_value = 0.0;} } } else { /*else it is categorical*/ if((int)*current_predictor1 == (int)knot1_value) {*function_value =1;} else {*function_value =0;} } if(predictor2 != 0) { if(knots_per_pred[predictor2-1]>=0) { if(knot2_index == 0) { *function_value= *function_value* (*current_predictor2); } else { if(*current_predictor2-knot2_value < 0.0) {*function_value =0.0;} else {*function_value =*function_value*(*current_predictor2-knot2_value);} } } else { /*else it is categorical*/ if((int)*current_predictor2 == (int)knot2_value) {*function_value =1;} else {*function_value =0;} } } function_value++; current_predictor1++; current_predictor2++; } mean = 0.0; SD = 1.0; standardise_array(function_values->matrix,cases,&mean,&SD); /*--put the info about the mean and standard deviation into the function--*/ new_column->function->SD=SD; new_column->function->mean=mean; /*--YtX inner product--------------------------*/ for(i = 0; i < responses; i++) { entry = 0.0; for(j=0;jmatrix[(i*cases)+j]) *function_values->matrix[j]; } new_column->data[i]=entry; } /*get XtX inner products by matrix multiplication*/ new_XtXcolumn->nrow=model_size; new_XtXcolumn->ncol=1; if(*weighted == TRUE) { temp_matrix->nrow=model_size; temp_matrix->ncol=cases; matrix_multiplication1(X_matrix,weight_matrix,temp_matrix,3); matrix_multiplication1(temp_matrix,function_values,new_XtXcolumn,0); } else { matrix_multiplication1(X_matrix,function_values,new_XtXcolumn,1); } for(i = 0; i < model_size; i++) { new_column->data[responses+i]=new_XtXcolumn->matrix[i]; } if(*weighted == TRUE) { temp_matrix->nrow=1; temp_matrix->ncol=cases; matrix_multiplication1(function_values, weight_matrix, temp_matrix, 3); matrix_multiplication1(temp_matrix, function_values, function_values_2, 0); } else { matrix_multiplication1(function_values, function_values, function_values_2, 1); } new_column->data[responses+model_size]=function_values_2->matrix[0]; return; } /*==============================================================*/ int new_candidate(int predictor1, int knot1_index, int predictor2, int knot2_index, struct basis_function_matrix *candidates, struct basis_function_matrix *model, struct matrix2 *YtXXtX_expanded) /*==============================================================*/ { /*Checks to see whether the function described by its predictor and knot indicies is already a candidate. If it is not then it adds the the candidate to the "candidates matrix" and also create a column in YtXXtX for it. This function is only called if the function 'in model" returns false when finding candidates */ struct basis_function_matrix* current_predictor; struct basis_function *current_function; struct basis_function *new_function; struct link *current_column; struct link *new_column; double *new_column_data; int i; current_predictor = candidates; if(interaction_specs_size >0) { for(i=0;ipredictor_index != predictor1) { current_predictor = current_predictor->next_predictor; } if(current_predictor->number_of_basis_functions ==0) { new_function= (struct basis_function *)calloc (1,sizeof(struct basis_function)); new_function->knot1_index = knot1_index; new_function->predictor1 = predictor1; new_function->predictor2 = predictor2; new_function->knot2_index = knot2_index; if(new_function->knot1_index == 0) new_function->knot1_value = 0.0; if(new_function->knot2_index == 0) new_function->knot2_value = 0.0; current_predictor->number_of_basis_functions = 1; current_predictor->functions = new_function; } else { current_function = current_predictor->functions; for(i=0;inumber_of_basis_functions;i++) { if(current_function->knot1_index == knot1_index && current_function->predictor2 == predictor2 && current_function->knot2_index == knot2_index) { return FALSE; } if(i != current_predictor->number_of_basis_functions-1) { current_function = current_function->link; } } new_function = (struct basis_function *)calloc (1,sizeof(struct basis_function)); current_function->link = new_function; new_function->knot1_index = knot1_index; new_function->predictor1 = predictor1; new_function->predictor2 = predictor2; new_function->knot2_index = knot2_index; if(new_function->knot1_index == 0) new_function->knot1_value = 0.0; if(new_function->knot2_index == 0) new_function->knot2_value = 0.0; /*the actual values of the knots are entered later*/ current_predictor->number_of_basis_functions++; } /*--add a column to YtXXtX_expanded--*/ current_column = YtXXtX_expanded->column_list; for(i=0;incol-1;i++) { current_column=current_column->next; } new_column = (struct link *)calloc(1,sizeof(struct link)); current_column->next = new_column; new_column_data = (double *)calloc(max_model_size+responses+1,sizeof(double)); new_column->data = new_column_data; new_column->function = new_function; YtXXtX_expanded->ncol++; return TRUE; } /*==============================================================*/ int find_best_candidate(struct matrix2 *YtXXtX_expanded, struct basis_function_matrix *model, struct basis_function_matrix *candidates, double *mesh) /*==============================================================*/ { /*- calculates the best candidate to add by adding it to the model and computing the "residual sum of squares". the function returns an index to the best candidate it finds*/ int number_of_candidates; int i, j, k,l,m,index; int nrow; double column_minder,row_minder; double E, E_inv; struct basis_function *model_function; struct link *YtXXtX_column; double Rao_D; struct link *current_column, *trailing_column; double RSS_so_far; double rss_for_model,gcv_for_model,gcv_so_far; int best_candidate; int candidate_found; RSS_so_far = -1; gcv_so_far = -1; best_candidate =0; candidate_found = FALSE; /*--computes XtX_inverse with new candidate added by Rao Linear algebra p33------*/ number_of_candidates = (YtXXtX_expanded->ncol)-model_size; XtX_newinverse->nrow=model_size+1; XtX_newinverse->ncol=model_size+1; for(i=0;icolumn_list; current_column = YtXXtX_expanded->column_list; for(k=0;knext; } Rao_B->nrow = model_size; Rao_B->ncol = 1; for(j=0;jmatrix[j]=current_column->data[j+responses]; } Rao_D = current_column->data[model_size+responses]; Rao_F->nrow = model_size; Rao_F->ncol = 1; matrix_multiplication1(XtX_inverse, Rao_B, Rao_F, 1); Rao_E->nrow=1; Rao_E->ncol=1; Rao_E->matrix = &E; matrix_multiplication1(Rao_B, Rao_F, Rao_E, 1); if(1/(Rao_D-Rao_E->matrix[0]) < tolerance || Rao_D-Rao_E->matrix[0] < tolerance) { trailing_column->next = current_column->next; i--; number_of_candidates--; YtXXtX_expanded->ncol--; } else { Rao_E->matrix[0] = Rao_D-Rao_E->matrix[0]; Rao_E_inverse->nrow=1; Rao_E_inverse->ncol=1; Rao_E_inverse->matrix = &E_inv; Rao_E_inverse->matrix[0]=1/Rao_E->matrix[0]; Rao_F_E_inverse->nrow = model_size; Rao_F_E_inverse->ncol = 1; matrix_multiplication1(Rao_F, Rao_E_inverse, Rao_F_E_inverse, 0); Rao_F_E_inverse_Ft ->nrow = model_size; Rao_F_E_inverse_Ft ->ncol = model_size; matrix_multiplication1(Rao_F,Rao_F_E_inverse,Rao_F_E_inverse_Ft,2); nrow = XtX_newinverse->nrow; for(j=0;jmatrix[(j*nrow)+k] = XtX_inverse->matrix[index] + Rao_F_E_inverse_Ft->matrix[index]; } } for(j=0;jmatrix[index] = -Rao_F_E_inverse->matrix[j]; index=(model_size*nrow) + j; XtX_newinverse->matrix[index]= -Rao_F_E_inverse->matrix[j]; } index = (model_size*nrow) + model_size; XtX_newinverse->matrix[index]=Rao_E_inverse->matrix[0]; /*---computes YtY-YtX(XtX)^1XtY--and- sums the diagonal to get the RSS------*/ /*---uses data matrix with nrow changed for Yt-----*/ /*- the column of the YtXXtX matrix corresponding to the candidate function being considered*/ /*switch the column of the candidate so it is adjacent to the model columns*/ switch_columns(model_size+i+1,model_size+1,YtXXtX_expanded); /*we wish to use only the YtX part of the YtXXtX matrix for the current model and the candidate being considered*/ column_minder=YtXXtX_expanded->ncol; row_minder=YtXXtX_expanded->nrow; YtXXtX_expanded->ncol=model_size+1; YtXXtX_expanded->nrow=responses; XtX_newinverseXtY->nrow = model_size+1; XtX_newinverseXtY->ncol = responses; matrix_multiplication2(XtX_newinverse, YtXXtX_expanded, XtX_newinverseXtY, 0); YtXXtX_newinverseXtY->nrow = responses; YtXXtX_newinverseXtY->ncol = responses; matrix_multiplication2(XtX_newinverseXtY, YtXXtX_expanded, YtXXtX_newinverseXtY, 1); YtXXtX_expanded->nrow=row_minder; YtXXtX_expanded->ncol=column_minder; rss_for_model =0.0; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; /* if the residual sum of squares is */ /* negative discard candidate */ if((YtY->matrix[j*(responses+1)] -YtXXtX_newinverseXtY->matrix[j*(responses+1)])<0.0) { rss_for_model = -1.0; j=responses; } } if(rss_for_model>=0.0) { for(j=0;jmatrix[j*(responses+1)] -YtXXtX_newinverseXtY->matrix[j*(responses+1)]) > rssgcv[(step_count)*(responses+1)+j]) { /* if the residual sum of squares go up for any reason*/ /* the candidate is rejected.*/ rss_for_model = -1; } } } if(rss_for_model > 0.0) { candidate_found =TRUE; if(RSS_so_far !=-1)/* if first candidate*/ { if(rss_for_model < RSS_so_far) { best_candidate = i; RSS_so_far = rss_for_model; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } } } else { best_candidate = i; RSS_so_far = rss_for_model; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } } /* gcv (or other criterion) is calculated, the best model in this call of the function if it is better than the best global model gets saved, in any case the gcv for this case is saved in rssgcv */ if(testset == FALSE) { gcv_for_model = (rss_for_model /weight_sum)/ ((1.0 - (GCVconstant*(model_size+1)/cases)) *(1.0 - (GCVconstant*(model_size+1)/cases))); } else { gcv_for_model = testset_RSS(YtXXtX_expanded,model_size+1); } if(model_size == 1 ) { if(GCV == -1)/*GCV was initially set to -1*/ { GCV = gcv_for_model; } } if(gcv_for_model < gcv_so_far || gcv_so_far==-1.0) { gcv_so_far = gcv_for_model; } if(GCV == -1.0 || gcv_for_model ncol*XtX_newinverseXtY->nrow);l++) { best_coefficents[l] = XtX_newinverseXtY->matrix[l]; } for(l=0;l<(XtX_newinverse->ncol);l++) { for(m=0;m<(XtX_newinverse->ncol);m++) { best_XtXinv[l+m*XtX_newinverse->ncol]= XtX_newinverse->matrix[l+m*XtX_newinverse->ncol]; } } *bestmodel_size = model_size+1; YtXXtX_column = YtXXtX_expanded->column_list; /*storing the best model */ for(l=0;lfunction; best_model[(l-1)*4]= model_function->predictor1; best_model[((l-1)*4)+1]=model_function->knot1_index; best_model[((l-1)*4)+2]=model_function->predictor2; best_model[((l-1)*4)+3]=model_function->knot2_index; best_model_sd_mean[((l-1)*2)]=model_function->SD; best_model_sd_mean[((l-1)*2)+1]=model_function->mean; } YtXXtX_column = YtXXtX_column->next; } } } /* switch 'candidates column' back to its original position*/ switch_columns(model_size+i+1,model_size+1,YtXXtX_expanded); } rssgcv[(step_count+1)*(responses+1)+responses]=gcv_so_far; } if (candidate_found != FALSE) { update_model(YtXXtX_expanded, model, candidates, best_candidate, mesh); rss_for_model =0.0; RSS_so_far =0.0; for(i =0;icolumn_list; for(i=0;inext; } candidates_column->data[model_size+responses] = 0.0; new_model_function = candidates_column->function; if(Verbose == TRUE) { printf("+ %d : %d ",model_size,new_model_function->predictor1); if(knots_per_pred[new_model_function->predictor1-1]>=0 && new_model_function->knot1_index != 0) { printf("%f ",new_model_function->knot1_value); } else { if(knots_per_pred[new_model_function->predictor1-1]>=0) {printf("NA ");} if(knots_per_pred[new_model_function->predictor1-1]<0) {printf("%d ",(int)new_model_function->knot1_value);} } if(new_model_function->predictor2 == 0) { printf("\n"); } else { if(new_model_function->knot2_index==0) {printf("%d NA\n",new_model_function->predictor2);} else {printf("%d %f\n",new_model_function->predictor2,new_model_function->knot1_value);} } } /*fflush(stdout);*/ predictor1 = new_model_function->predictor1; predictor2 = new_model_function->predictor2; /*---------find function in candidates----------*/ current_candidate_predictor = candidates; while(current_candidate_predictor->predictor_index != predictor1) { current_candidate_predictor = current_candidate_predictor->next_predictor; } current_candidate_function = current_candidate_predictor->functions; trailing_function = current_candidate_predictor->functions; while(current_candidate_function != new_model_function) { trailing_function = current_candidate_function; current_candidate_function = current_candidate_function->link; } /* take out of the candidates matrix*/ if(trailing_function == current_candidate_function) { current_candidate_predictor->functions = current_candidate_function->link; } else { trailing_function->link = current_candidate_function->link; } current_candidate_predictor->number_of_basis_functions--; /* add to model matrix*/ current_model_predictor=model; while(current_model_predictor->predictor_index != predictor1) { current_model_predictor = current_model_predictor->next_predictor; } current_model_function=current_model_predictor->functions; for(i=0;inumber_of_basis_functions-1;i++) { current_model_function= current_model_function->link; } if(current_model_predictor->number_of_basis_functions==0) { current_model_predictor->functions= new_model_function; } else { current_model_function->link=new_model_function; } current_model_predictor->number_of_basis_functions++; knot1_index = new_model_function->knot1_index; knot2_index = new_model_function->knot2_index; /*-----------------hierarchy terms----------------------------*/ if(predictor2 == 0) { if(knot1_index !=0) { order_keeper1[predictor1-1]++; } } else { if(knot1_index !=0 || knot2_index != 0) { order_keeper2[((predictor1-1)*predictors)+predictor2-1]++; order_keeper2[((predictor2-1)*predictors)+predictor1-1]++; } if(knot1_index ==0 || knot2_index == 0) { order_keeper1[predictor1-1]++; order_keeper1[predictor2-1]++; } if(max_knots != 0) { if(knot1_index !=0) { order_keeper3[(predictor1-1)*max_knots+knot1_index-1]++; } if(knot2_index != 0) { order_keeper3[(predictor2-1)*max_knots+knot2_index-1]++; } } } /*------------------update X_matrix----------------------------*/ /*calculate the basis function values at the points of the data matrix*/ X_matrix->ncol++; X_matrix_ptr = &X_matrix->matrix[(model_size-1)*cases]; /*--move to row of predictor of interest in data matrix---*/ current_predictor_values1 = &data_matrix->matrix[((predictor1-1)+responses)*cases]; if(predictor2!=0) { current_predictor_values2 = &data_matrix->matrix[((predictor2-1)+responses)*cases]; } /*---find the knot values for the basis function--*/ if(new_model_function->knot1_index !=0) { knot1_value = new_model_function->knot1_value; } if(new_model_function->knot2_index !=0) { knot2_value = new_model_function->knot2_value; } for(i=0;i= 0) { *X_matrix_ptr=current_predictor_values1[i]; if(knot1_index != 0) { *X_matrix_ptr = *X_matrix_ptr - knot1_value; if(*X_matrix_ptr <0){*X_matrix_ptr = 0;} } } else { /*else it is categorical*/ if((int)current_predictor_values1[i] == (int)knot1_value) {*X_matrix_ptr =1;} else {*X_matrix_ptr =0;} } if(predictor2 != 0) { if(knots_per_pred[predictor2-1] >= 0) { if(knot2_index == 0) { *X_matrix_ptr = *X_matrix_ptr * current_predictor_values2[i]; } else { if(current_predictor_values2[i] - knot2_value <0.0) {*X_matrix_ptr = 0.0;} else { *X_matrix_ptr = *X_matrix_ptr*(current_predictor_values2[i] - knot2_value); } } } else { /*else it is categorical*/ if((int)current_predictor_values2[i] != (int)knot2_value) {*X_matrix_ptr=0;} } } X_matrix_ptr++; } /*--Standardise the new column of the X matrix---------------------*/ for(i=0;imatrix[((model_size-1)*cases)+i]= (X_matrix->matrix[((model_size-1)*cases)+i]- new_model_function->mean)/new_model_function->SD; } /*--computes XtX_inverse with new candidate added by Rao Linear algebra p33------*/ Rao_B->nrow = model_size-1; Rao_B->ncol = 1; for(j=0;jmatrix[j]=candidates_column->data[j+responses]; } Rao_D = candidates_column->data[model_size-1+responses]; /*-----------------------------------------------------------------*/ Rao_F->nrow = model_size-1; Rao_F->ncol = 1; matrix_multiplication1(XtX_inverse,Rao_B,Rao_F,1); /*-----------------------------------------------------------------*/ Rao_E->matrix = &E; matrix_multiplication1(Rao_B,Rao_F,Rao_E,1); Rao_E->matrix[0] = Rao_D-Rao_E->matrix[0]; Rao_E_inverse->nrow =1; Rao_E_inverse->ncol = 1; Rao_E_inverse->matrix = &E_inv; Rao_E_inverse->matrix[0]=1/Rao_E->matrix[0]; Rao_F_E_inverse->ncol= 1; Rao_F_E_inverse->nrow = model_size-1; matrix_multiplication1(Rao_F,Rao_E_inverse,Rao_F_E_inverse,0); Rao_F_E_inverse_Ft->nrow = model_size-1; Rao_F_E_inverse_Ft->ncol = model_size-1; matrix_multiplication1(Rao_F,Rao_F_E_inverse,Rao_F_E_inverse_Ft,2); XtX_inverse->nrow++; XtX_inverse->ncol++; nrow = XtX_inverse->nrow; for(j=0;j<(model_size-1);j++) { for(k=0;k<(model_size-1);k++) { XtX_newinverse->matrix[(model_size*j)+k] = XtX_inverse->matrix[((model_size-1)*j)+k] + Rao_F_E_inverse_Ft->matrix[((model_size-1)*j)+k]; } } for(i=0;imatrix[i]= XtX_newinverse->matrix[i]; } for(j=0;jmatrix[index] = -Rao_F_E_inverse->matrix[j]; index=(model_size-1)*nrow + j; XtX_inverse->matrix[index]= -Rao_F_E_inverse->matrix[j]; } index = (model_size*model_size)-1; XtX_inverse->matrix[index]=Rao_E_inverse->matrix[0]; /*---------------------------------------------------------------- update YtXXtX_expanded the new inner product between candidate and new function in model are calculated and inserted, the number of rows grows by 1 ----------------------------------------------------------------*/ YtXXtX_expanded->nrow++; nrow = YtXXtX_expanded->nrow; current_column = YtXXtX_expanded->column_list; for(i=0;idata[nrow-2]= candidates_column->data[responses+i]; current_column = current_column->next; } function_values->nrow=cases; function_values->ncol=1; /*--move to first candidate column of the matrix to update it--*/ current_column = candidates_column->next; /*move through all the candidates in the YtXXtX matrix */ for(i=0;incol-model_size;i++) { current_function = current_column->function; knot1_index = current_function->knot1_index; predictor1=current_function->predictor1; knot2_index = current_function->knot2_index; predictor2=current_function->predictor2; /*--move to row of predictor of interest in data matrix---*/ current_predictor_values1 = &data_matrix->matrix[(current_function->predictor1+responses-1) *cases]; /*--evaluate the functions at each case-- to calculate inner products--*/ if(predictor2!=0) { current_predictor_values2= &data_matrix->matrix[(current_function->predictor2+ responses-1) *cases]; } function_value = function_values->matrix; if(knot1_index==0) { for(j=0;j0) { function_value[j] = matrix_entry; } else { function_value[j] = 0.0; } } } } } if(predictor2!=0) { if(knot2_index==0) { for(j=0;j0) { function_value[j] = function_value[j] *(current_predictor_values2[j]- knot1_value); } else { function_value[j] = 0.0; } } } } } standardise_array(function_value,cases,&dummy1,&dummy2); matrix_entry =0.0; for(j=0;jmatrix[(model_size-1)*cases+j] *weight_matrix->matrix[j]); } else { matrix_entry = matrix_entry+ function_value[j]*X_matrix->matrix[(model_size-1)*cases+j]; } } current_column->data[YtXXtX_expanded->nrow-1]= current_column->data[YtXXtX_expanded->nrow-2]; current_column->data[YtXXtX_expanded->nrow-2]= matrix_entry; current_column = current_column->next; } } /*================================================================*/ int reduce_model(struct matrix2 *YtXXtX_expanded, struct basis_function_matrix *model) /*================================================================*/ { /*function reduces model by one function. Takes out each candidate in turn, computes RSS and the one resulting in the lowest RSS is taken out and everthing is updated. some function may have been specified to stay in the model*/ int i,j,k,l,m; int predictor1, predictor2,knot1_index,knot2_index; int column_minder; int best_candidate; int candidate_to_remove; double *switch_matrix; double rss_for_model,rss_so_far=0,gcv_for_model,gcv_so_far ; struct link *current_predictor_col, *trailing_column; struct basis_function *current_predictor; struct basis_function *trailing_function; struct basis_function *current_model_function; struct basis_function *discard_model_function; struct basis_function_matrix* current_model_predictor; int cant_remove;/*boolean*/ gcv_so_far = -1.0; steps[(step_count+1)*2]=0; steps[(step_count+1)*2+1]=model_size-1; best_candidate = -1; XtX_newinverse->nrow = model_size-1; XtX_newinverse->ncol = model_size-1; new_X_matrix->nrow=cases; new_X_matrix->ncol =model_size-1; current_predictor_col = YtXXtX_expanded->column_list; candidate_to_remove = FALSE; for(i=1;inext; current_predictor=current_predictor_col->function; /*------------------------------- Check is the function must remain in the model because of input specifications -------------------------------*/ if(not_remove_size > 0) { for(j=0;jpredictor1 && startmodel[((not_remove_specs[j]-1)*4)+1] == current_predictor->knot1_index && startmodel[((not_remove_specs[j]-1)*4)+2] == current_predictor->predictor2 && startmodel[((not_remove_specs[j]-1)*4)+3] == current_predictor->knot2_index) { cant_remove= TRUE; } } } /*------------------------------------------------------------ Certain order of removal rules are followed if 1 knot is 0 check that there is no double knot interaction if 1 term with knot check that it is not part of interaction if linear check that there no knot terms or interaction etc. -------------------------------------------------------------*/ if(current_predictor->predictor2 ==0) { if(current_predictor->knot1_index == 0) { if(order_keeper1[current_predictor->predictor1-1]>0) {cant_remove = TRUE;} } else { if(max_knots != 0) { if(order_keeper3[(current_predictor->predictor1-1) *max_knots+current_predictor->knot1_index-1]>0) {cant_remove = TRUE;} } } } else { if(current_predictor->knot1_index == 0) { for(j=0;jpredictor1-1]);j++) { if(in_model(current_predictor->predictor1, j+1, current_predictor->predictor2, current_predictor->knot2_index, model)) {cant_remove = TRUE;} } } if(current_predictor->knot2_index == 0) { for(j=0;jpredictor2-1]);j++) { if(in_model(current_predictor->predictor1, current_predictor->knot1_index, current_predictor->predictor2, j+1, model)) { cant_remove = TRUE; } } } /* }*/ } if(cant_remove != TRUE) { candidate_to_remove = TRUE; /*-------create new XtX_inverse_without the one predictor----------*/ /*using backwards version of the method in addition stage*/ for(j = 0;j=i){l=j+1;}else{l=j;} if(k>=i){m=k+1;}else{m=k;} XtX_newinverse->matrix[(j*(model_size-1))+k] = XtX_inverse->matrix[(l*model_size)+m] - ((XtX_inverse->matrix[l*(model_size)+i]) *(XtX_inverse->matrix[i*(model_size)+m]) /XtX_inverse->matrix[(i)*(model_size)+i]); } } /*making a new X matrix*/ for(j=0;jmatrix[(j*cases)+k] = X_matrix->matrix[l]; } } /*calculating RSS */ column_minder = data_matrix->ncol; data_matrix->ncol = responses; YtX->nrow = responses; YtX->ncol = model_size-1; matrix_multiplication1(data_matrix,new_X_matrix,YtX,1); data_matrix->ncol = column_minder; XtX_newinverseXtY->nrow=model_size-1; XtX_newinverseXtY->ncol= responses; matrix_multiplication1(XtX_newinverse,YtX,XtX_newinverseXtY,2); YtXXtX_newinverseXtY->nrow=responses; YtXXtX_newinverseXtY->ncol=responses; matrix_multiplication1(YtX,XtX_newinverseXtY,YtXXtX_newinverseXtY,0); rss_for_model =0.0; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } if(testset == FALSE) { gcv_for_model = ( rss_for_model/weight_sum)/ ((1.0 - (GCVconstant*(model_size-1)/cases))* (1.0 - (GCVconstant*(model_size-1)/cases))); } else { gcv_for_model =testset_RSS(YtXXtX_expanded,model_size-1); } if(best_candidate !=-1)/* if it is not the first iteration*/ { if(rss_for_model < rss_so_far) { best_candidate = i; rss_so_far = rss_for_model; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } } } else { best_candidate = i; rss_so_far = rss_for_model; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } } if(gcv_so_far == -1.0) { gcv_so_far = gcv_for_model; } else { if(gcv_for_model < gcv_so_far) { gcv_so_far = gcv_for_model; } } if(gcv_for_model < GCV) { for(j=0;j<(XtX_newinverseXtY->ncol*XtX_newinverseXtY->nrow);j++) { best_coefficents[j] = XtX_newinverseXtY->matrix[j]; } for(l=0;l<(XtX_newinverse->ncol);l++) { for(m=0;m<(XtX_newinverse->ncol);m++) { best_XtXinv[l+m*XtX_newinverse->ncol]= XtX_newinverse->matrix[l+m*XtX_newinverse->ncol]; } } } } } if(candidate_to_remove == FALSE) { return FALSE; } step_count++; rssgcv[(step_count)*(responses+1)+responses]=gcv_so_far; /*-----------------------------------------------------------------*/ /*candidate to remove is found and now it will be removed*/ /*reduce model*/ /*-----------------------------------------------------------------*/ /*-find the function in the YtXXtX matrix by its index--*/ current_predictor_col = YtXXtX_expanded->column_list; for(i=0;inext; } trailing_column->next = current_predictor_col->next; discard_model_function=current_predictor_col->function; if(Verbose == TRUE) { printf("- %d : %d ",model_size-1,discard_model_function->predictor1); if(knots_per_pred[discard_model_function->predictor1-1]>=0 && discard_model_function->knot1_index != 0) { printf("%f ",discard_model_function->knot1_value); } else { if(knots_per_pred[discard_model_function->predictor1-1]>=0) {printf("NA ");} if(knots_per_pred[discard_model_function->predictor1-1]<0) {printf("%d ",(int)discard_model_function->knot1_value);} } if(discard_model_function->predictor2 == 0) { printf("\n"); } else { if(discard_model_function->knot2_index==0) {printf("%d NA\n",discard_model_function->predictor2);} else {printf("%d %f\n",discard_model_function->predictor2,discard_model_function->knot1_value);} } } /*fflush(stdout);*/ /*-find the function in the model matrix and remove it--*/ predictor1 = discard_model_function->predictor1; current_model_predictor = model; while(current_model_predictor->predictor_index != predictor1) { current_model_predictor = current_model_predictor->next_predictor; } current_model_function = current_model_predictor->functions; trailing_function = current_model_predictor->functions; while(current_model_function != discard_model_function) { trailing_function = current_model_function; current_model_function = current_model_function->link; } if(trailing_function== current_model_function) { current_model_predictor->functions= current_model_function->link; } else { trailing_function->link = current_model_function->link; } current_model_predictor->number_of_basis_functions--; /*-update the matrix for deletion order-*/ knot1_index = discard_model_function->knot1_index; predictor2 = discard_model_function->predictor2; knot2_index = discard_model_function->knot2_index; if(predictor2 ==0) { if(knot1_index!=0) { order_keeper1[predictor1-1]--; } } else { if(knot1_index !=0 || knot2_index !=0) { order_keeper2[((predictor1-1)*predictors)+predictor2-1]--; order_keeper2[((predictor2-1)*predictors)+predictor1-1]--; } if(knot1_index ==0 || knot2_index ==0) { order_keeper1[predictor1-1]--; order_keeper1[predictor2-1]--; } if(max_knots != 0) { if(knot1_index!= 0) { order_keeper3[(predictor1-1)*max_knots+knot1_index-1]--; } if(knot2_index!= 0) { order_keeper3[(predictor2-1)*max_knots+knot2_index-1]--; } } } /*--------------update---X_matrix--------------------------------*/ for(j=1;jmatrix[(j*cases)+k] = X_matrix->matrix[l]; } } switch_matrix =new_X_matrix->matrix; new_X_matrix->matrix = X_matrix->matrix; X_matrix->matrix = switch_matrix; X_matrix->ncol--; /*--------------update---XtX_invmatrix--------------------------------*/ XtX_inverse->ncol--; XtX_inverse->nrow--; if(*weighted == TRUE) { temp_matrix->nrow = model_size; temp_matrix->ncol = cases; matrix_multiplication1(X_matrix,weight_matrix,temp_matrix,3); matrix_multiplication1(temp_matrix,X_matrix,XtX_inverse,0); } else { matrix_multiplication1(X_matrix,X_matrix,XtX_inverse,1); } invert_matrix(XtX_inverse); model_size--; /*-save new global best model if this iteration produced better gcv*/ if(gcv_so_far < GCV) { GCV = gcv_so_far; *bestmodel_size = model_size; current_predictor_col = YtXXtX_expanded->column_list; for(l=0;lfunction; best_model[(l-1)*4]= current_predictor->predictor1; best_model[((l-1)*4)+1]=current_predictor->knot1_index; best_model[((l-1)*4)+2]=current_predictor->predictor2; best_model[((l-1)*4)+3]=current_predictor->knot2_index; best_model_sd_mean[(l-1)*2]= current_predictor->SD; best_model_sd_mean[((l-1)*2)+1]= current_predictor->mean; } current_predictor_col = current_predictor_col->next; } } return TRUE; } /*----------------------------------------------------------*/ int initial_model(struct basis_function_matrix *model, struct matrix2 **YtXXtX_expanded) /*---------------------------------------------------------*/ { /* Set up the initial model, by default it is a model containing only the intercept*/ int i,j,l,m,predictor_1,predictor_2,ok,knot_1_index,knot_2_index,mesh_index; double *X_ptr , knot_1_value,knot_2_value; double rss_for_model; struct link *YtXXtX_column; int column_minder; double mean,SD; double *means,*SDs; struct basis_function_matrix* current_predictor; struct basis_function *current_function; struct basis_function *new_function; ok = 0; if(model_size != 1) { means= (double *)calloc (model_size-1,sizeof(double)); SDs= (double *)calloc (model_size-1,sizeof(double)); } /*create matrix to hold YtX and XtX for model and candidates */ (*YtXXtX_expanded)= create_matrix2(max_model_size+responses+1,model_size); (*YtXXtX_expanded)->nrow = responses+model_size+1; (*YtXXtX_expanded)->ncol = model_size; X_matrix=create_matrix1(0,0); X_matrix->ncol=model_size; X_matrix->nrow = cases; X_matrix->matrix= (double *)calloc (max_model_size*cases,sizeof(double)); X_ptr = X_matrix->matrix; /* fit the constant function (intercept) over all predictors */ for(i=0;i= 0) { *X_ptr = data_matrix->matrix[((responses+(predictor_1-1))*cases)+j]; if(knot_1_index != 0) { *X_ptr = *X_ptr - knot_1_value; if (*X_ptr < 0.0){*X_ptr = 0.0;} } } else { /*else it is categorical*/ if((int)data_matrix->matrix[((responses+(predictor_1-1))*cases)+j] == (int)knot_1_value) {*X_ptr =1;} else {*X_ptr =0;} } if(predictor_2 != 0) { if(knots_per_pred[predictor_2-1] >= 0) { if(knot_2_index == 0) { *X_ptr = *X_ptr*data_matrix ->matrix[((responses+(predictor_2-1))*cases)+j]; } else { if(data_matrix->matrix[((responses+(predictor_2-1))*cases)+j] - knot_2_value < 0.0) { *X_ptr =0.0; } else { *X_ptr = *X_ptr * (data_matrix->matrix[((responses+(predictor_2-1)) *cases)+j]- knot_2_value); } } } else { /*else it is categorical*/ if((int)data_matrix->matrix[((responses+(predictor_1-1))*cases)+j] != (int)knot_2_value) {*X_ptr =0;} } } X_ptr++; } mean=0.0; SD=1.0; standardise_array(X_ptr-cases,cases,&mean,&SD); means[i]=mean; SDs[i] = SD; } /*--Making the XtY bit of the the YtXXtX matrix------------*/ if(*weighted == TRUE) { column_minder = data_matrix->ncol; data_matrix->ncol = responses; YtX->nrow = responses; YtX->ncol = model_size; temp_matrix->ncol = cases; temp_matrix->nrow = responses; matrix_multiplication1(data_matrix,weight_matrix,temp_matrix,3); YtX->nrow = responses; YtX->ncol = model_size; matrix_multiplication1(temp_matrix,X_matrix,YtX,0); } else { column_minder = data_matrix->ncol; data_matrix->ncol = responses; YtX->nrow = responses; YtX->ncol = model_size; matrix_multiplication1(data_matrix,X_matrix,YtX,1); } data_matrix->ncol = column_minder; YtXXtX_column = (*YtXXtX_expanded)->column_list; for(i=0;i data[j]=YtX->matrix[(i*responses)+j]; } if(i != model_size-1){YtXXtX_column = YtXXtX_column->next;} } /*--Making the XtX bit of the the YtXXtX matrix------------*/ XtX_inverse->ncol = model_size; XtX_inverse->nrow = model_size; if(*weighted == TRUE) { temp_matrix->nrow = model_size; temp_matrix->ncol = cases; matrix_multiplication1(X_matrix,weight_matrix,temp_matrix,3); matrix_multiplication1(temp_matrix,X_matrix,XtX_inverse,0); } else { matrix_multiplication1(X_matrix,X_matrix,XtX_inverse,1); } YtXXtX_column = (*YtXXtX_expanded)->column_list; for(i=0;i data[j+responses]=XtX_inverse->matrix[(i*model_size)+j]; } if(i != model_size-1){YtXXtX_column = YtXXtX_column->next;} } /*---Inserting the model into the model functions structure--*/ YtXXtX_column = (*YtXXtX_expanded)->column_list; for(i=0;i next; current_predictor = model; predictor_1= startmodel[(i*4)]; knot_1_index = startmodel[(i*4)+1]; predictor_2 = startmodel[(i*4)+2]; knot_2_index = startmodel[(i*4)+3]; while(current_predictor->predictor_index != predictor_1) { current_predictor = current_predictor->next_predictor; } if(current_predictor->number_of_basis_functions ==0) { new_function= (struct basis_function *)calloc (1,sizeof(struct basis_function)); new_function->knot1_index = knot_1_index; new_function->knot1_value =startknots[i*2]; new_function->predictor1 = predictor_1; new_function->predictor2 = predictor_2; new_function->knot2_index = knot_2_index; new_function->knot2_value =startknots[(i*2)+1]; new_function->SD = SDs[i]; new_function->mean= means[i]; current_predictor->number_of_basis_functions = 1; current_predictor->functions = new_function; } else { /* move to the end of the column to add a new basis function*/ current_function = current_predictor->functions; for(j=0;jnumber_of_basis_functions;j++) { if(j != current_predictor->number_of_basis_functions-1) { current_function = current_function->link; } } new_function = (struct basis_function *)calloc (1,sizeof(struct basis_function)); current_function->link = new_function; new_function->knot1_index = knot_1_index; new_function->predictor1 = predictor_1; new_function->predictor2 = predictor_2; new_function->knot2_index = knot_2_index; new_function->knot1_value =startknots[i*2]; new_function->knot2_value =startknots[(i*2)+1]; new_function->SD = SDs[i]; new_function->mean =means[i]; current_predictor->number_of_basis_functions++; } YtXXtX_column->function = new_function; /*-updating the matricies which control the order in which new candidates can be added-*/ if(predictor_2 == 0) { if(knot_1_index !=0) { order_keeper1[predictor_1-1]++; } } else { if(knot_1_index !=0 || knot_2_index != 0) { order_keeper2[((predictor_1-1)*predictors)+predictor_2-1]++; order_keeper2[((predictor_2-1)*predictors)+predictor_1-1]++; } if(knot_1_index ==0 || knot_2_index == 0) { order_keeper1[predictor_1-1]++; order_keeper1[predictor_2-1]++; } if(max_knots != 0) { if(knot_1_index !=0) { order_keeper3[(predictor_1-1)*max_knots+knot_1_index-1]++; } if(knot_2_index != 0) { order_keeper3[(predictor_2-1)*max_knots+knot_2_index-1]++; } } } } /*--inverting the XtX_matrix----------------*/ if(model_size ==1) { XtX_inverse->matrix[0] = 1/XtX_inverse->matrix[0]; } else { /*--Lapack inversion for indefinite double precision real symmetric matricies----two step--factorisation and inversion-------*/ ok = invert_matrix(XtX_inverse); } if (ok == 0) { /*computes YtY-YtX(XtX)^1XtY--and- sums the diagonal to get the RSS*/ /*uses data matrix with number of rows changed to be number of responses for Yt. */ (*YtXXtX_expanded)->nrow=responses; XtX_newinverseXtY->nrow = model_size; XtX_newinverseXtY->ncol = responses; matrix_multiplication2(XtX_inverse, *YtXXtX_expanded, XtX_newinverseXtY, 0); YtXXtX_newinverseXtY->nrow = responses; YtXXtX_newinverseXtY->ncol = responses; matrix_multiplication2(XtX_newinverseXtY, *YtXXtX_expanded, YtXXtX_newinverseXtY, 1); (*YtXXtX_expanded)->nrow=responses+model_size+1; rss_for_model =0.0; for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } for(j=0;jmatrix[j*(responses+1)] +YtY->matrix[j*(responses+1)]; } if(testset == FALSE) { GCV = (rss_for_model /weight_sum)/ ((1.0 - (GCVconstant*(model_size)/cases)) *(1.0 - (GCVconstant*(model_size)/cases))); } else { GCV = testset_RSS((*YtXXtX_expanded),model_size); } rssgcv[responses]=GCV; for(j=0;j<(XtX_newinverseXtY->nrow*XtX_newinverseXtY->ncol);j++) { best_coefficents[j] = XtX_newinverseXtY->matrix[j]; } for(l=0;l<(XtX_newinverse->ncol);l++) { for(m=0;m<(XtX_newinverse->ncol);m++) { best_XtXinv[l+m*XtX_newinverse->ncol]= XtX_newinverse->matrix[l+m*XtX_newinverse->ncol]; } } *bestmodel_size = model_size; YtXXtX_column = (*YtXXtX_expanded)->column_list; /*storing the best model */ for(i=0;ifunction; best_model[(i-1)*4]= current_function->predictor1; best_model[((i-1)*4)+1]=current_function->knot1_index; best_model[((i-1)*4)+2]=current_function->predictor2; best_model[((i-1)*4)+3]=current_function->knot2_index; best_model_sd_mean[(i-1)*2]=current_function->SD; best_model_sd_mean[((i-1)*2)+1]=current_function->mean; } YtXXtX_column = YtXXtX_column->next; } steps[0]=1; steps[1]=model_size; } return ok; } /*==============================================================*/ void standardise_array(double *numbers,int length_of_list,double *mean,double *SD) /*==============================================================*/ { /*---------------------------------------------------------------- Finds the mean and standard deviation of a list of numbers and returns the array standard normalised, with mean and standard deviatioN ------------------------------------------------------------------*/ int i,mina,maxi; *mean = 0; mina = numbers[0]; maxi = numbers[0]; for(i=0;i maxi){maxi=numbers[i];} } if((mina-maxi)*(mina-maxi)>=1) { *mean = (*mean)/length_of_list; for(i=0;i=0) { prereq =FALSE; for(j = 0;j< model_size-1;j++) { /*---checks that linear is present when initial model includes knot term-*/ if((startmodel[(j*4)] == startmodel[i*2] && startmodel[(j*4)+1] ==0) && startmodel[(j*4)+2] ==0) { prereq = TRUE; } } if(prereq == FALSE){return 2;} } } /*-checks the prerequisites for compound functions and whether the initial model contains the same terms more than once--*/ for(i = 0;i< model_size-1;i++) { for(j = 0;j< model_size-1;j++) { if(i !=j) { if(startmodel[(i*4)] == startmodel[(j*4)] && startmodel[(i*4)+1] == startmodel[(j*4)+1] && startmodel[(i*4)+2] == startmodel[(j*4)+2] && startmodel[(i*4)+3] == startmodel[(j*4)+3]) { /*if both are linear*/ if(startmodel[(i*4)+1] == 0 && startmodel[(i*4)+3]==0) { return 1; } /* if knots are the same */ if(startmodel[(i*4)+1] == 0 && startmodel[(i*4)+3] != 0) { if(startknots[(i*2)+1] == startknots[(j*2)+1]) { return 1; } } if(startmodel[(i*4)+1] != 0 && startmodel[(i*4)+3] == 0) { if(startknots[i*2] == startknots[j*2]) { return 1; } } if(startmodel[(i*4)] ==startmodel[(j*4)+2] && startmodel[(i*4)+1] == startmodel[(j*4)+3] && startmodel[(i*4)+2] == startmodel[(j*4)] && startmodel[(i*4)+3] == startmodel[(j*4)+1]) { if(startmodel[(i*4)+1] == 0 && startmodel[(i*4)+3]==0) { return 1; } if(startmodel[(i*4)+1] == 0 && startmodel[(i*4)+3] != 0) { if(startknots[(i*2)+1] == startknots[j*2]) { return 1; } } if(startmodel[(i*4)+1] != 0 && startmodel[(i*4)+3] == 0) { if(startknots[i*2] == startknots[(j*2)+1]) { return 1; } } } } } } /* a basis function with 2 linear terms then must have each linear term also in model*/ if((startmodel[(i*4)] !=0 && startmodel[(i*4)+1] == 0) && startmodel[(i*4)+2] != 0 && startmodel[(i*4)+3] ==0) { prereq = FALSE; prereq2 = FALSE; for(j = 0;j< model_size-1;j++) { if(startmodel[(j*4)] ==startmodel[(i*4)] && startmodel[(j*4)+1] == 0 && startmodel[(j*4)+2] == 0 && startmodel[(j*4)+3] == 0) {prereq = TRUE;} if(startmodel[(j*4)] ==startmodel[(i*4)+2] && startmodel[(j*4)+1] == 0 && startmodel[(j*4)+2] == 0 && startmodel[(j*4)+3] == 0) {prereq2 = TRUE;} } if(prereq == FALSE || prereq2 == FALSE) {return 3;} } /*if a model has two knoted terms the two linear term model must be present, also each two term with both predictors and one knot*/ if((startmodel[(i*4)] !=0 && startmodel[(i*4)+2] != 0) && startmodel[(i*4)+1] != 0 && startmodel[(i*4)+3] !=0) { prereq=FALSE; for(j = 0;j< model_size-1;j++) { if((startmodel[(i*4)] == startmodel[j*4] && startmodel[(j*4)+1] == 0 && startmodel[(i*4)+2] == startmodel[(j*4)+2] && startmodel[(i*4)+3] ==0) || (startmodel[(i*4)] == startmodel[(j*4)+2] && startmodel[(i*4)+3] ==0 && startmodel[(i*4)+2] == startmodel[j*4] && startmodel[(j*4)+1] == 0 )) { prereq = TRUE; } } if(prereq==FALSE){return 3;} prereq=FALSE; prereq2=FALSE; for(j = 0;j< model_size-1;j++) { if(startmodel[(i*4)] == startmodel[j*4] && 0 ==startmodel[(j*4)+1] && startmodel[(i*4)+2] == startmodel[(j*4)+2] && startmodel[(i*4)+3] == startmodel[(j*4)+3]) { prereq = TRUE; } if(startmodel[(i*4)] == startmodel[(j*4)+2] && 0 == startmodel[(j*4)+3] && startmodel[(i*4)+2] == startmodel[(j*4)] && startmodel[(i*4)+3] == startmodel[(j*4)+1]) { prereq = TRUE; } if(startmodel[(i*4)] == startmodel[j*4] &&startmodel[(i*4)+1] == startmodel[(j*4)+1] &&startmodel[(i*4)+2] == startmodel[(j*4)+2] && 0 == startmodel[(j*4)+3]) { prereq2 = TRUE; } if(startmodel[(i*4)] == startmodel[(j*4)+2] &&startmodel[(i*4)+1] == startmodel[(j*4)+3] &&startmodel[(i*4)+2] == startmodel[j*4] && 0 == startmodel[(j*4)+1]) { prereq2 = TRUE; } } if(prereq == FALSE || prereq2 == FALSE){return 3;} } } } return 0; } /*==============================================================*/ double testset_RSS(struct matrix2 *YtXXtX_expanded,int model_size ) /*==============================================================*/ { /*computes test set RSS, coefficients must be transformed as they apply to the standarised data */ double totalRSS,fitted; struct basis_function *model_function; struct link *YtXXtX_column; double standardise_const; int i,j,k,cases; double temp_value1,temp_value2,x; cases = testset_matrix->nrow; YtXXtX_column = YtXXtX_expanded->column_list; for(i=0;i<(XtX_newinverseXtY->ncol*XtX_newinverseXtY->nrow);i++) { coefficents[i]=XtX_newinverseXtY->matrix[i]; } /*get mean and standard deviations */ for(i=0;inext; model_sd_mean[i*2] = YtXXtX_column->function->SD; model_sd_mean[(i*2)+1] = YtXXtX_column->function->mean; } /*untransform the transformed data*/ for(i=0;icolumn_list; standardise_const=0.0; for(j=1;jnext; standardise_const = standardise_const +(coefficents[j+(i*model_size)]*model_sd_mean[((j-1)*2)+1])/model_sd_mean[(j-1)*2]; } coefficents[i*(model_size)] = coefficents[i*model_size]-standardise_const; } for(i=0;ifunction->predictor1] < 0 && YtXXtX_column->function->predictor2 == 0 )) { coefficents[j+(i*model_size)] =coefficents[j+(i*model_size)] /model_sd_mean[(j-1)*2]; } } } for(i=0;icolumn_list; fitted = coefficents[i*(XtX_newinverseXtY->nrow)]; for(k=0;knext; model_function = YtXXtX_column->function; temp_value2 = 1; if(knots_per_pred[model_function->predictor1-1]>=0) { temp_value1 = testset_matrix->matrix[((responses+model_function->predictor1-1)*cases)+j]; if(model_function->knot1_index != 0) { x=temp_value1 - model_function->knot1_value; if(x>0){temp_value1 = x;}else{temp_value1 = 0.0;} } } else { if((int)testset_matrix->matrix[((responses+model_function->predictor1-1)*cases)+j] == (int)model_function->knot1_value) {temp_value1 =1.0;}else{temp_value1 = 0.0;} } if(model_function->predictor2 != 0) { temp_value2 = testset_matrix->matrix[((responses+model_function->predictor2-1)*cases)+j]; if(model_function->knot1_index != 0) { x=temp_value2 - model_function->knot2_value; if(x>0){temp_value2 = x;}else{temp_value2 = 0.0;} } } fitted = fitted + (temp_value1*temp_value2*coefficents[i*(XtX_newinverseXtY->nrow)+k+1]); } if(classification != TRUE) { tset_RSS[i] = tset_RSS[i] + (fitted - testset_matrix->matrix[i*cases+j])* (fitted - testset_matrix->matrix[i*cases+j]); } else { /* in classification the class is the index of the largest response for a case*/ if(i==0) { response_max[j] = fitted; response_class[j] = 0; } else { if(fitted>response_max[j]) { response_max[j]=fitted; response_class[j] = i; } } } } } totalRSS = 0.0; if(classification == TRUE) { for(j=0;jmatrix[response_class[j]*cases+j]) ==0 ) { totalRSS = totalRSS + 1; } } } else { if(testset_weighted == TRUE) { for(i=0;incol = ncol; new_matrix->nrow = nrow; return(new_matrix); } /*==============================================================*/ struct matrix2 *create_matrix2(int nrow, int ncol) /*==============================================================*/ { /*--------------------------------------------------------------- Creates a matrix which is a linked list of columns. Each link also has a linkl to the model or candidate matrix as each column corresponds to one basis function -----------------------------------------------------------------*/ int i; struct matrix2 *new_matrix; struct link *column; struct link *current_link=0; new_matrix = (struct matrix2 *)calloc(1,sizeof(struct matrix2)); for(i=0;idata =(double *)calloc(nrow,sizeof(double)); column->next = current_link; current_link= column; } new_matrix->ncol = ncol; new_matrix->nrow = nrow; new_matrix->column_list = current_link; return(new_matrix); } /*==============================================================*/ void switch_columns(int col1, int col2, struct matrix2 *object_matrix) /*==============================================================*/ { /*---------------------------------------------------------------- Switching rows means just swapping the pointers to the rows in the list that binds the column together ----------------------------------------------------------------*/ int i; struct link *current_column1, *current_column2; struct basis_function *function_holder; double *ptr_holder; if(col1==col2){return;} current_column1= object_matrix->column_list; current_column2= object_matrix->column_list; for(i=0;inext; } for(i=0;inext; } ptr_holder= current_column1->data; function_holder = current_column1->function; current_column1->function = current_column2->function; current_column1->data= current_column2->data; current_column2->data=ptr_holder; current_column2->function = function_holder; } /*============================================================== void print_matrix1(struct matrix1 *object_matrix ) ============================================================== { int i,j,nrow,ncol; nrow = object_matrix->nrow; ncol = object_matrix->ncol; for(i=0;imatrix[(j*nrow)+i]); } printf("\n"); } } */ /*============================================================== void print_matrix2(struct matrix2 *object_matrix ) ============================================================== { int i,j,k,nrow,ncol; struct link *current_column; nrow = object_matrix->nrow; ncol = object_matrix->ncol; current_column = object_matrix->column_list; for(i=0;inext; } printf("%f\t",current_column->data[i]); current_column = object_matrix->column_list; } printf("\n"); } } */ /*==============================================================*/ void matrix_multiplication1(struct matrix1 *object_matrixA, struct matrix1 *object_matrixB, struct matrix1 *result, int flag) /*==============================================================*/ { /*---------------------------------------------------------------- Flag: 0 for AB (regular multiplicaion (A,B,0)->AB 1 for AB (multiply transpose of A by B (A,B,1)->AtB) 2 for AB (multiply A by transpose of B (A,B,2)->ABt 3 for special multiplication AW where W is a weight matrix n*n with only the n diagonal elements stored ----------------------------------------------------------------*/ int i,j,k,nrowA,nrowB,ncolA,ncolB; double *A_pointer; double *B_pointer; double product; nrowA = object_matrixA-> nrow; ncolA = object_matrixA-> ncol; nrowB = object_matrixB-> nrow; ncolB = object_matrixB-> ncol; if(flag==1) { /*if(nrowA != nrowB) { printf("Multiplication error \n"); exit(1); } if(ncolA != result->nrow || ncolB != result->ncol) { printf("Multiplication error \n"); exit(1); }*/ for(i=0;imatrix; for(j=0;jmatrix[i*nrowA]; for(k=0;kmatrix[j*ncolA+i]=product; } } } if(flag==0) { A_pointer = object_matrixA->matrix; B_pointer = object_matrixB->matrix; /*if(ncolA != nrowB) { printf("Multiplication error3 \n"); exit(1); } if(nrowA != result->nrow || ncolB != result->ncol) { printf("Multiplication error4 \n"); exit(1); } */ for(i=0;imatrix; for(j=0;jmatrix[i+result->nrow*j]=product; } } } if(flag==2) { /* if(ncolA != ncolB) { printf("Multiplication error4 \n"); exit(1); } if(nrowA != result->nrow || nrowB != result->ncol) { printf("Multiplication error \n"); exit(1); } */ A_pointer = object_matrixA->matrix; B_pointer = object_matrixB->matrix; for(i=0;imatrix[i+nrowA*j]=product; } } } if(flag==3) { /* if(nrowA != nrowB) { printf("Multiplication error \n"); exit(1); } if(ncolA != result->nrow || ncolB != result->ncol) { printf("Multiplication error\n"); exit(1); } */ B_pointer = object_matrixB->matrix; for(i=0;imatrix[i*nrowA]; k=0; for(j=0;jmatrix[j*ncolA+i]=product; } } } return ; } /*==============================================================*/ void matrix_multiplication2(struct matrix1 *object_matrixA, struct matrix2 *object_matrixB, struct matrix1 *result, int flag) /*==============================================================*/ { /* multipling a matrix1 with a matrix2 flag for whether matrix2 is transposed or not*/ int i,j,k,l,nrowA,nrowB,ncolA,ncolB; double *A_pointer; double *B_pointer; double product; struct link *column_pointer; nrowA = object_matrixA-> nrow; ncolA = object_matrixA-> ncol; nrowB = object_matrixB-> nrow; ncolB = object_matrixB-> ncol; if(flag==0) { /*if(ncolA != ncolB) { printf("Multiplication error \n"); exit(1); } if (nrowA != result->nrow || nrowB != result->ncol) { printf("Multiplication error \n"); exit(1); } */ A_pointer=object_matrixA->matrix; for(i=0;icolumn_list; for(l=0;lnext; } B_pointer=column_pointer->data; product=product+(A_pointer[i+nrowA*k]*B_pointer[j]); } result->matrix[i+(nrowA*j)]=product; } } } else { /*if(nrowA != ncolB) { printf("Multiplication error\n"); exit(1); }*/ /*if(nrowB != result->nrow || ncolA != result->ncol) { printf("Multiplication error \n"); exit(1); } */ for(i=0;imatrix; for(j=0;jcolumn_list; for(l=0;lnext; } B_pointer=column_pointer->data; product = product + B_pointer[i]*A_pointer[0]; A_pointer++; } result->matrix[i+(nrowB*j)]=product; } } } return ; } /*============================================================== void print_functions(struct basis_function_matrix *functions_matrix) ============================================================== { int i,j; struct basis_function_matrix *current_predictor; struct basis_function *function; current_predictor = functions_matrix; for(i=0;inumber_of_basis_functions != 0) {function = current_predictor->functions;} for(j=0;jnumber_of_basis_functions;j++) { printf("%d\t%d\t%f\t%d\t%d\t%f\n",function->predictor1, function->knot1_index, function->knot1_value, function->predictor2, function->knot2_index, function->knot2_value); if(j!=current_predictor->number_of_basis_functions-1) {function = function->link;} } current_predictor= current_predictor->next_predictor; } } */ /*---------------------------------------------------------------*/ logical lsame_(char *ca, char *cb) { /* -- LAPACK auxiliary routine (version 2.0) -- Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd., Courant Institute, Argonne National Lab, and Rice University September 30, 1994 Purpose ======= LSAME returns .TRUE. if CA is the same letter as CB regardless of case. Arguments ========= CA (input) CHARACTER*1 CB (input) CHARACTER*1 CA and CB specify the single characters to be compared. ===================================================================== Test if the characters are equal */ /* System generated locals */ logical ret_val; /* Local variables */ static int inta, intb, zcode; ret_val = *(unsigned char *)ca == *(unsigned char *)cb; if (ret_val) { return ret_val; } /* Now test for equivalence if both characters are alphabetic. */ zcode = 'Z'; /* Use 'Z' rather than 'A' so that ASCII can be detected on Prime machines, on which ICHAR returns a value with bit 8 set. ICHAR('A') on Prime machines returns 193 which is the same as ICHAR('A') on an EBCDIC machine. */ inta = *(unsigned char *)ca; intb = *(unsigned char *)cb; if (zcode == 90 || zcode == 122) { /* ASCII is assumed - ZCODE is the ASCII code of either lower o r upper case 'Z'. */ if (inta >= 97 && inta <= 122) { inta += -32; } if (intb >= 97 && intb <= 122) { intb += -32; } } else if (zcode == 233 || zcode == 169) { /* EBCDIC is assumed - ZCODE is the EBCDIC code of either lower or upper case 'Z'. */ if (inta >= 129 && inta <= 137 || inta >= 145 && inta <= 153 || inta >= 162 && inta <= 169) { inta += 64; } if (intb >= 129 && intb <= 137 || intb >= 145 && intb <= 153 || intb >= 162 && intb <= 169) { intb += 64; } } else if (zcode == 218 || zcode == 250) { /* ASCII is assumed, on Prime machines - ZCODE is the ASCII cod e plus 128 of either lower or upper case 'Z'. */ if (inta >= 225 && inta <= 250) { inta += -32; } if (intb >= 225 && intb <= 250) { intb += -32; } } ret_val = inta == intb; /* RETURN End of LSAME */ return ret_val; } /* lsame_ */ /* Subroutine */ int xerbla_(char *srname, int *info) { /* -- LAPACK auxiliary routine (version 2.0) -- Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd., Courant Institute, Argonne National Lab, and Rice University September 30, 1994 Purpose ======= XERBLA is an error handler for the LAPACK routines. It is called by an LAPACK routine if an input parameter has an invalid value. A message is printed and execution stops. Installers may consider modifying the STOP statement in order to call system-specific exception-handling facilities. Arguments ========= SRNAME (input) CHARACTER*6 The name of the routine which called XERBLA. INFO (input) INT The position of the invalid parameter in the parameter list of the calling routine. ===================================================================== */ printf("** On entry to %6s, parameter number %2i had an illegal value\n", srname, *info); /* End of XERBLA */ return 0; } /* xerbla_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ int idamax_(int *n, double *dx, int *incx) { /* System generated locals */ int ret_val, i__1; double d__1; /* Local variables */ static double dmax__; static int i, ix; /* finds the index of element having max. absolute value. jack dongarra, linpack, 3/11/78. modified 3/93 to return if incx .le. 0. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ #define DX(I) dx[(I)-1] ret_val = 0; if (*n < 1 || *incx <= 0) { return ret_val; } ret_val = 1; if (*n == 1) { return ret_val; } if (*incx == 1) { goto L20; } /* code for increment not equal to 1 */ ix = 1; dmax__ = abs(DX(1)); ix += *incx; i__1 = *n; for (i = 2; i <= *n; ++i) { if ((d__1 = DX(ix), abs(d__1)) <= dmax__) { goto L5; } ret_val = i; dmax__ = (d__1 = DX(ix), abs(d__1)); L5: ix += *incx; /* L10: */ } return ret_val; /* code for increment equal to 1 */ L20: dmax__ = abs(DX(1)); i__1 = *n; for (i = 2; i <= *n; ++i) { if ((d__1 = DX(i), abs(d__1)) <= dmax__) { goto L30; } ret_val = i; dmax__ = (d__1 = DX(i), abs(d__1)); L30: ; } return ret_val; } /* idamax_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int dswap_(int *n, double *dx, int *incx, double *dy, int *incy) { /* System generated locals */ int i__1; /* Local variables */ static int i, m; static double dtemp; static int ix, iy, mp1; /* interchanges two vectors. uses unrolled loops for increments equal one. jack dongarra, linpack, 3/11/78. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ #define DY(I) dy[(I)-1] if (*n <= 0) { return 0; } if (*incx == 1 && *incy == 1) { goto L20; } /* code for unequal increments or equal increments not equal to 1 */ ix = 1; iy = 1; if (*incx < 0) { ix = (-(*n) + 1) * *incx + 1; } if (*incy < 0) { iy = (-(*n) + 1) * *incy + 1; } i__1 = *n; for (i = 1; i <= *n; ++i) { dtemp = DX(ix); DX(ix) = DY(iy); DY(iy) = dtemp; ix += *incx; iy += *incy; /* L10: */ } return 0; /* code for both increments equal to 1 clean-up loop */ L20: m = *n % 3; if (m == 0) { goto L40; } i__1 = m; for (i = 1; i <= m; ++i) { dtemp = DX(i); DX(i) = DY(i); DY(i) = dtemp; /* L30: */ } if (*n < 3) { return 0; } L40: mp1 = m + 1; i__1 = *n; for (i = mp1; i <= *n; i += 3) { dtemp = DX(i); DX(i) = DY(i); DY(i) = dtemp; dtemp = DX(i + 1); DX(i + 1) = DY(i + 1); DY(i + 1) = dtemp; dtemp = DX(i + 2); DX(i + 2) = DY(i + 2); DY(i + 2) = dtemp; /* L50: */ } return 0; } /* dswap_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int dspr_(char *uplo, int *n, double *alpha, double *x, int *incx, double *ap) { /* System generated locals */ int i__1, i__2; /* Local variables */ static int info; static double temp; static int i, j, k; extern logical lsame_(char *, char *); static int kk, ix, jx, kx; extern /* Subroutine */ int xerbla_(char *, int *); /* Purpose ======= DSPR performs the symmetric rank 1 operation A := alpha*x*x' + A, where alpha is a real scalar, x is an n element vector and A is an n by n symmetric matrix, supplied in packed form. Parameters ========== UPLO - CHARACTER*1. On entry, UPLO specifies whether the upper or lower triangular part of the matrix A is supplied in the packed array AP as follows: UPLO = 'U' or 'u' The upper triangular part of A is supplied in AP. UPLO = 'L' or 'l' The lower triangular part of A is supplied in AP. Unchanged on exit. N - INT. On entry, N specifies the order of the matrix A. N must be at least zero. Unchanged on exit. ALPHA - DOUBLE PRECISION. On entry, ALPHA specifies the scalar alpha. Unchanged on exit. X - DOUBLE PRECISION array of dimension at least ( 1 + ( n - 1 )*abs( INCX ) ). Before entry, the incremented array X must contain the n element vector x. Unchanged on exit. INCX - INT. On entry, INCX specifies the increment for the elements of X. INCX must not be zero. Unchanged on exit. AP - DOUBLE PRECISION array of DIMENSION at least ( ( n*( n + 1 ) )/2 ). Before entry with UPLO = 'U' or 'u', the array AP must contain the upper triangular part of the symmetric matrix packed sequentially, column by column, so that AP( 1 ) contains a( 1, 1 ), AP( 2 ) and AP( 3 ) contain a( 1, 2 ) and a( 2, 2 ) respectively, and so on. On exit, the array AP is overwritten by the upper triangular part of the updated matrix. Before entry with UPLO = 'L' or 'l', the array AP must contain the lower triangular part of the symmetric matrix packed sequentially, column by column, so that AP( 1 ) contains a( 1, 1 ), AP( 2 ) and AP( 3 ) contain a( 2, 1 ) and a( 3, 1 ) respectively, and so on. On exit, the array AP is overwritten by the lower triangular part of the updated matrix. Level 2 Blas routine. -- Written on 22-October-1986. Jack Dongarra, Argonne National Lab. Jeremy Du Croz, Nag Central Office. Sven Hammarling, Nag Central Office. Richard Hanson, Sandia National Labs. Test the input parameters. Parameter adjustments Function Body */ #define AP(I) ap[(I)-1] #define X(I) x[(I)-1] info = 0; if (! lsame_(uplo, "U") && ! lsame_(uplo, "L")) { info = 1; } else if (*n < 0) { info = 2; } else if (*incx == 0) { info = 5; } if (info != 0) { xerbla_("DSPR ", &info); return 0; } /* Quick return if possible. */ if (*n == 0 || *alpha == 0.) { return 0; } /* Set the start point in X if the increment is not unity. */ if (*incx <= 0) { kx = 1 - (*n - 1) * *incx; } else if (*incx != 1) { kx = 1; } /* Start the operations. In this version the elements of the array AP are accessed sequentially with one pass through AP. */ kk = 1; if (lsame_(uplo, "U")) { /* Form A when upper triangle is stored in AP. */ if (*incx == 1) { i__1 = *n; for (j = 1; j <= *n; ++j) { if (X(j) != 0.) { temp = *alpha * X(j); k = kk; i__2 = j; for (i = 1; i <= j; ++i) { AP(k) += X(i) * temp; ++k; /* L10: */ } } kk += j; /* L20: */ } } else { jx = kx; i__1 = *n; for (j = 1; j <= *n; ++j) { if (X(jx) != 0.) { temp = *alpha * X(jx); ix = kx; i__2 = kk + j - 1; for (k = kk; k <= kk+j-1; ++k) { AP(k) += X(ix) * temp; ix += *incx; /* L30: */ } } jx += *incx; kk += j; /* L40: */ } } } else { /* Form A when lower triangle is stored in AP. */ if (*incx == 1) { i__1 = *n; for (j = 1; j <= *n; ++j) { if (X(j) != 0.) { temp = *alpha * X(j); k = kk; i__2 = *n; for (i = j; i <= *n; ++i) { AP(k) += X(i) * temp; ++k; /* L50: */ } } kk = kk + *n - j + 1; /* L60: */ } } else { jx = kx; i__1 = *n; for (j = 1; j <= *n; ++j) { if (X(jx) != 0.) { temp = *alpha * X(jx); ix = jx; i__2 = kk + *n - j; for (k = kk; k <= kk+*n-j; ++k) { AP(k) += X(ix) * temp; ix += *incx; /* L70: */ } } jx += *incx; kk = kk + *n - j + 1; /* L80: */ } } } return 0; /* End of DSPR . */ } /* dspr_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int dscal_(int *n, double *da, double *dx, int *incx) { /* System generated locals */ int i__1, i__2; /* Local variables */ static int i, m, nincx, mp1; /* scales a vector by a constant. uses unrolled loops for increment equal to one. jack dongarra, linpack, 3/11/78. modified 3/93 to return if incx .le. 0. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ if (*n <= 0 || *incx <= 0) { return 0; } if (*incx == 1) { goto L20; } /* code for increment not equal to 1 */ nincx = *n * *incx; i__1 = nincx; i__2 = *incx; for (i = 1; *incx < 0 ? i >= nincx : i <= nincx; i += *incx) { DX(i) = *da * DX(i); /* L10: */ } return 0; /* code for increment equal to 1 clean-up loop */ L20: m = *n % 5; if (m == 0) { goto L40; } i__2 = m; for (i = 1; i <= m; ++i) { DX(i) = *da * DX(i); /* L30: */ } if (*n < 5) { return 0; } L40: mp1 = m + 1; i__2 = *n; for (i = mp1; i <= *n; i += 5) { DX(i) = *da * DX(i); DX(i + 1) = *da * DX(i + 1); DX(i + 2) = *da * DX(i + 2); DX(i + 3) = *da * DX(i + 3); DX(i + 4) = *da * DX(i + 4); /* L50: */ } return 0; } /* dscal_ */ /* Subroutine */ int dlaev2_(double *a, double *b, double *c, double *rt1, double *rt2, double *cs1, double *sn1) { /* -- LAPACK auxiliary routine (version 2.0) -- Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd., Courant Institute, Argonne National Lab, and Rice University October 31, 1992 Purpose ======= DLAEV2 computes the eigendecomposition of a 2-by-2 symmetric matrix [ A B ] [ B C ]. On return, RT1 is the eigenvalue of larger absolute value, RT2 is the eigenvalue of smaller absolute value, and (CS1,SN1) is the unit right eigenvector for RT1, giving the decomposition [ CS1 SN1 ] [ A B ] [ CS1 -SN1 ] = [ RT1 0 ] [-SN1 CS1 ] [ B C ] [ SN1 CS1 ] [ 0 RT2 ]. Arguments ========= A (input) DOUBLE PRECISION The (1,1) element of the 2-by-2 matrix. B (input) DOUBLE PRECISION The (1,2) element and the conjugate of the (2,1) element of the 2-by-2 matrix. C (input) DOUBLE PRECISION The (2,2) element of the 2-by-2 matrix. RT1 (output) DOUBLE PRECISION The eigenvalue of larger absolute value. RT2 (output) DOUBLE PRECISION The eigenvalue of smaller absolute value. CS1 (output) DOUBLE PRECISION SN1 (output) DOUBLE PRECISION The vector (CS1, SN1) is a unit right eigenvector for RT1. Further Details =============== RT1 is accurate to a few ulps barring over/underflow. RT2 may be inaccurate if there is massive cancellation in the determinant A*C-B*B; higher precision or correctly rounded or correctly truncated arithmetic would be needed to compute RT2 accurately in all cases. CS1 and SN1 are accurate to a few ulps barring over/underflow. Overflow is possible only if RT1 is within a factor of 5 of overflow. Underflow is harmless if the input data is 0 or exceeds underflow_threshold / macheps. ===================================================================== Compute the eigenvalues */ /* System generated locals */ double d__1; /* Builtin functions */ double sqrt(double); /* Local variables */ static double acmn, acmx, ab, df, cs, ct, tb, sm, tn, rt, adf, acs; static int sgn1, sgn2; sm = *a + *c; df = *a - *c; adf = abs(df); tb = *b + *b; ab = abs(tb); if (abs(*a) > abs(*c)) { acmx = *a; acmn = *c; } else { acmx = *c; acmn = *a; } if (adf > ab) { /* Computing 2nd power */ d__1 = ab / adf; rt = adf * sqrt(d__1 * d__1 + 1.); } else if (adf < ab) { /* Computing 2nd power */ d__1 = adf / ab; rt = ab * sqrt(d__1 * d__1 + 1.); } else { /* Includes case AB=ADF=0 */ rt = ab * sqrt(2.); } if (sm < 0.) { *rt1 = (sm - rt) * .5; sgn1 = -1; /* Order of execution important. To get fully accurate smaller eigenvalue, next line needs to be executed in higher precision. */ *rt2 = acmx / *rt1 * acmn - *b / *rt1 * *b; } else if (sm > 0.) { *rt1 = (sm + rt) * .5; sgn1 = 1; /* Order of execution important. To get fully accurate smaller eigenvalue, next line needs to be executed in higher precision. */ *rt2 = acmx / *rt1 * acmn - *b / *rt1 * *b; } else { /* Includes case RT1 = RT2 = 0 */ *rt1 = rt * .5; *rt2 = rt * -.5; sgn1 = 1; } /* Compute the eigenvector */ if (df >= 0.) { cs = df + rt; sgn2 = 1; } else { cs = df - rt; sgn2 = -1; } acs = abs(cs); if (acs > ab) { ct = -tb / cs; *sn1 = 1. / sqrt(ct * ct + 1.); *cs1 = ct * *sn1; } else { if (ab == 0.) { *cs1 = 1.; *sn1 = 0.; } else { tn = -cs / tb; *cs1 = 1. / sqrt(tn * tn + 1.); *sn1 = tn * *cs1; } } if (sgn1 == sgn2) { tn = *cs1; *cs1 = -(*sn1); *sn1 = tn; } return 0; /* End of DLAEV2 */ } /* dlaev2_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int drot_(int *n, double *dx, int *incx, double *dy, int *incy, double *c, double *s) { /* System generated locals */ int i__1; /* Local variables */ static int i; static double dtemp; static int ix, iy; /* applies a plane rotation. jack dongarra, linpack, 3/11/78. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ if (*n <= 0) { return 0; } if (*incx == 1 && *incy == 1) { goto L20; } /* code for unequal increments or equal increments not equal to 1 */ ix = 1; iy = 1; if (*incx < 0) { ix = (-(*n) + 1) * *incx + 1; } if (*incy < 0) { iy = (-(*n) + 1) * *incy + 1; } i__1 = *n; for (i = 1; i <= *n; ++i) { dtemp = *c * DX(ix) + *s * DY(iy); DY(iy) = *c * DY(iy) - *s * DX(ix); DX(ix) = dtemp; ix += *incx; iy += *incy; /* L10: */ } return 0; /* code for both increments equal to 1 */ L20: i__1 = *n; for (i = 1; i <= *n; ++i) { dtemp = *c * DX(i) + *s * DY(i); DY(i) = *c * DY(i) - *s * DX(i); DX(i) = dtemp; /* L30: */ } return 0; } /* drot_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int dcopy_(int *n, double *dx, int *incx, double *dy, int *incy) { /* System generated locals */ int i__1; /* Local variables */ static int i, m, ix, iy, mp1; /* copies a vector, x, to a vector, y. uses unrolled loops for increments equal to one. jack dongarra, linpack, 3/11/78. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ if (*n <= 0) { return 0; } if (*incx == 1 && *incy == 1) { goto L20; } /* code for unequal increments or equal increments not equal to 1 */ ix = 1; iy = 1; if (*incx < 0) { ix = (-(*n) + 1) * *incx + 1; } if (*incy < 0) { iy = (-(*n) + 1) * *incy + 1; } i__1 = *n; for (i = 1; i <= *n; ++i) { DY(iy) = DX(ix); ix += *incx; iy += *incy; /* L10: */ } return 0; /* code for both increments equal to 1 clean-up loop */ L20: m = *n % 7; if (m == 0) { goto L40; } i__1 = m; for (i = 1; i <= m; ++i) { DY(i) = DX(i); /* L30: */ } if (*n < 7) { return 0; } L40: mp1 = m + 1; i__1 = *n; for (i = mp1; i <= *n; i += 7) { DY(i) = DX(i); DY(i + 1) = DX(i + 1); DY(i + 2) = DX(i + 2); DY(i + 3) = DX(i + 3); DY(i + 4) = DX(i + 4); DY(i + 5) = DX(i + 5); DY(i + 6) = DX(i + 6); /* L50: */ } return 0; } /* dcopy_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ /* Subroutine */ int dspmv_(char *uplo, int *n, double *alpha, double *ap, double *x, int *incx, double *beta, double *y, int *incy) { /* System generated locals */ int i__1, i__2; /* Local variables */ static int info; static double temp1, temp2; static int i, j, k; extern logical lsame_(char *, char *); static int kk, ix, iy, jx, jy, kx, ky; extern /* Subroutine */ int xerbla_(char *, int *); /* Purpose ======= DSPMV performs the matrix-vector operation y := alpha*A*x + beta*y, where alpha and beta are scalars, x and y are n element vectors and A is an n by n symmetric matrix, supplied in packed form. Parameters ========== UPLO - CHARACTER*1. On entry, UPLO specifies whether the upper or lower triangular part of the matrix A is supplied in the packed array AP as follows: UPLO = 'U' or 'u' The upper triangular part of A is supplied in AP. UPLO = 'L' or 'l' The lower triangular part of A is supplied in AP. Unchanged on exit. N - INT. On entry, N specifies the order of the matrix A. N must be at least zero. Unchanged on exit. ALPHA - DOUBLE PRECISION. On entry, ALPHA specifies the scalar alpha. Unchanged on exit. AP - DOUBLE PRECISION array of DIMENSION at least ( ( n*( n + 1 ) )/2 ). Before entry with UPLO = 'U' or 'u', the array AP must contain the upper triangular part of the symmetric matrix packed sequentially, column by column, so that AP( 1 ) contains a( 1, 1 ), AP( 2 ) and AP( 3 ) contain a( 1, 2 ) and a( 2, 2 ) respectively, and so on. Before entry with UPLO = 'L' or 'l', the array AP must contain the lower triangular part of the symmetric matrix packed sequentially, column by column, so that AP( 1 ) contains a( 1, 1 ), AP( 2 ) and AP( 3 ) contain a( 2, 1 ) and a( 3, 1 ) respectively, and so on. Unchanged on exit. X - DOUBLE PRECISION array of dimension at least ( 1 + ( n - 1 )*abs( INCX ) ). Before entry, the incremented array X must contain the n element vector x. Unchanged on exit. INCX - INT. On entry, INCX specifies the increment for the elements of X. INCX must not be zero. Unchanged on exit. BETA - DOUBLE PRECISION. On entry, BETA specifies the scalar beta. When BETA is supplied as zero then Y need not be set on input. Unchanged on exit. Y - DOUBLE PRECISION array of dimension at least ( 1 + ( n - 1 )*abs( INCY ) ). Before entry, the incremented array Y must contain the n element vector y. On exit, Y is overwritten by the updated vector y. INCY - INT. On entry, INCY specifies the increment for the elements of Y. INCY must not be zero. Unchanged on exit. Level 2 Blas routine. -- Written on 22-October-1986. Jack Dongarra, Argonne National Lab. Jeremy Du Croz, Nag Central Office. Sven Hammarling, Nag Central Office. Richard Hanson, Sandia National Labs. Test the input parameters. Parameter adjustments Function Body */ #define Y(I) y[(I)-1] info = 0; if (! lsame_(uplo, "U") && ! lsame_(uplo, "L")) { info = 1; } else if (*n < 0) { info = 2; } else if (*incx == 0) { info = 6; } else if (*incy == 0) { info = 9; } if (info != 0) { xerbla_("DSPMV ", &info); return 0; } /* Quick return if possible. */ if (*n == 0 || *alpha == 0. && *beta == 1.) { return 0; } /* Set up the start points in X and Y. */ if (*incx > 0) { kx = 1; } else { kx = 1 - (*n - 1) * *incx; } if (*incy > 0) { ky = 1; } else { ky = 1 - (*n - 1) * *incy; } /* Start the operations. In this version the elements of the array AP are accessed sequentially with one pass through AP. First form y := beta*y. */ if (*beta != 1.) { if (*incy == 1) { if (*beta == 0.) { i__1 = *n; for (i = 1; i <= *n; ++i) { Y(i) = 0.; /* L10: */ } } else { i__1 = *n; for (i = 1; i <= *n; ++i) { Y(i) = *beta * Y(i); /* L20: */ } } } else { iy = ky; if (*beta == 0.) { i__1 = *n; for (i = 1; i <= *n; ++i) { Y(iy) = 0.; iy += *incy; /* L30: */ } } else { i__1 = *n; for (i = 1; i <= *n; ++i) { Y(iy) = *beta * Y(iy); iy += *incy; /* L40: */ } } } } if (*alpha == 0.) { return 0; } kk = 1; if (lsame_(uplo, "U")) { /* Form y when AP contains the upper triangle. */ if (*incx == 1 && *incy == 1) { i__1 = *n; for (j = 1; j <= *n; ++j) { temp1 = *alpha * X(j); temp2 = 0.; k = kk; i__2 = j - 1; for (i = 1; i <= j-1; ++i) { Y(i) += temp1 * AP(k); temp2 += AP(k) * X(i); ++k; /* L50: */ } Y(j) = Y(j) + temp1 * AP(kk + j - 1) + *alpha * temp2; kk += j; /* L60: */ } } else { jx = kx; jy = ky; i__1 = *n; for (j = 1; j <= *n; ++j) { temp1 = *alpha * X(jx); temp2 = 0.; ix = kx; iy = ky; i__2 = kk + j - 2; for (k = kk; k <= kk+j-2; ++k) { Y(iy) += temp1 * AP(k); temp2 += AP(k) * X(ix); ix += *incx; iy += *incy; /* L70: */ } Y(jy) = Y(jy) + temp1 * AP(kk + j - 1) + *alpha * temp2; jx += *incx; jy += *incy; kk += j; /* L80: */ } } } else { /* Form y when AP contains the lower triangle. */ if (*incx == 1 && *incy == 1) { i__1 = *n; for (j = 1; j <= *n; ++j) { temp1 = *alpha * X(j); temp2 = 0.; Y(j) += temp1 * AP(kk); k = kk + 1; i__2 = *n; for (i = j + 1; i <= *n; ++i) { Y(i) += temp1 * AP(k); temp2 += AP(k) * X(i); ++k; /* L90: */ } Y(j) += *alpha * temp2; kk += *n - j + 1; /* L100: */ } } else { jx = kx; jy = ky; i__1 = *n; for (j = 1; j <= *n; ++j) { temp1 = *alpha * X(jx); temp2 = 0.; Y(jy) += temp1 * AP(kk); ix = jx; iy = jy; i__2 = kk + *n - j; for (k = kk + 1; k <= kk+*n-j; ++k) { ix += *incx; iy += *incy; Y(iy) += temp1 * AP(k); temp2 += AP(k) * X(ix); /* L110: */ } Y(jy) += *alpha * temp2; jx += *incx; jy += *incy; kk += *n - j + 1; /* L120: */ } } } return 0; /* End of DSPMV . */ } /* dspmv_ */ /* -- translated by f2c (version 19940927). You must link the resulting object file with the libraries: -lf2c -lm (in that order) */ double ddot_(int *n, double *dx, int *incx, double *dy, int *incy) { /* System generated locals */ int i__1; double ret_val; /* Local variables */ static int i, m; static double dtemp; static int ix, iy, mp1; /* forms the dot product of two vectors. uses unrolled loops for increments equal to one. jack dongarra, linpack, 3/11/78. modified 12/3/93, array(1) declarations changed to array(*) Parameter adjustments Function Body */ ret_val = 0.; dtemp = 0.; if (*n <= 0) { return ret_val; } if (*incx == 1 && *incy == 1) { goto L20; } /* code for unequal increments or equal increments not equal to 1 */ ix = 1; iy = 1; if (*incx < 0) { ix = (-(*n) + 1) * *incx + 1; } if (*incy < 0) { iy = (-(*n) + 1) * *incy + 1; } i__1 = *n; for (i = 1; i <= *n; ++i) { dtemp += DX(ix) * DY(iy); ix += *incx; iy += *incy; /* L10: */ } ret_val = dtemp; return ret_val; /* code for both increments equal to 1 clean-up loop */ L20: m = *n % 5; if (m == 0) { goto L40; } i__1 = m; for (i = 1; i <= m; ++i) { dtemp += DX(i) * DY(i); /* L30: */ } if (*n < 5) { goto L60; } L40: mp1 = m + 1; i__1 = *n; for (i = mp1; i <= *n; i += 5) { dtemp = dtemp + DX(i) * DY(i) + DX(i + 1) * DY(i + 1) + DX(i + 2) * DY(i + 2) + DX(i + 3) * DY(i + 3) + DX(i + 4) * DY(i + 4); /* L50: */ } L60: ret_val = dtemp; return ret_val; } /* ddot_ */ int invert_matrix(struct matrix1 *object_matrix) { char flag; int i,j,dim,k; int ok,info; int *int_array; double *packed_matrix,*workspace; ok =0; dim = object_matrix->nrow; /*--pack top half of this matrix into array for inversion routine-*/ packed_matrix= (double *)calloc(1+(dim*(dim+1))/2,sizeof(double)); k=-1; for(i=0;imatrix[(i*dim)+j]; } } int_array = (int *)calloc(dim+1,sizeof(int)); flag = 'U'; info=0; /*--factorisation step -----------------------------------*/ ok = dsptrf_(&flag,&dim,packed_matrix,int_array,&info); /*--inversion step----------------------------------------*/ workspace = (double *)calloc(dim*dim+1,sizeof(double)); ok = dsptri_(&flag,&dim,packed_matrix,int_array,workspace,&info); /*--unpacking into back into matrix ----------------------*/ k=0; for(j=0;jmatrix[(j*dim)+i]=packed_matrix[i + j*(j+1)/2]; } } for(j=1;jmatrix[j+(i*dim)]=object_matrix->matrix[(j*dim)+i]; } } return ok; } int dsptrf_(char *uplo, int *n, double *ap, int * ipiv, int *info) { /* -- LAPACK routine (version 2.0) -- Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd., Courant Institute, Argonne National Lab, and Rice University March 31, 1993 Purpose ======= DSPTRF computes the factorization of a real symmetric matrix A stored in packed format using the Bunch-Kaufman diagonal pivoting method: A = U*D*U**T or A = L*D*L**T where U (or L) is a product of permutation and unit upper (lower) triangular matrices, and D is symmetric and block diagonal with 1-by-1 and 2-by-2 diagonal blocks. Arguments ========= UPLO (input) CHARACTER*1 = 'U': Upper triangle of A is stored; = 'L': Lower triangle of A is stored. N (input) INT The order of the matrix A. N >= 0. AP (input/output) DOUBLE PRECISION array, dimension (N*(N+1)/2) On entry, the upper or lower triangle of the symmetric matrix A, packed columnwise in a linear array. The j-th column of A is stored in the array AP as follows: if UPLO = 'U', AP(i + (j-1)*j/2) = A(i,j) for 1<=i<=j; if UPLO = 'L', AP(i + (j-1)*(2n-j)/2) = A(i,j) for j<=i<=n. On exit, the block diagonal matrix D and the multipliers used to obtain the factor U or L, stored as a packed triangular matrix overwriting A (see below for further details). IPIV (output) INT array, dimension (N) Details of the interchanges and the block structure of D. If IPIV(k) > 0, then rows and columns k and IPIV(k) were interchanged and D(k,k) is a 1-by-1 diagonal block. If UPLO = 'U' and IPIV(k) = IPIV(k-1) < 0, then rows and columns k-1 and -IPIV(k) were interchanged and D(k-1:k,k-1:k) is a 2-by-2 diagonal block. If UPLO = 'L' and IPIV(k) = IPIV(k+1) < 0, then rows and columns k+1 and -IPIV(k) were interchanged and D(k:k+1,k:k+1) is a 2-by-2 diagonal block. INFO (output) INT = 0: successful exit < 0: if INFO = -i, the i-th argument had an illegal value > 0: if INFO = i, D(i,i) is exactly zero. The factorization has been completed, but the block diagonal matrix D is exactly singular, and division by zero will occur if it is used to solve a system of equations. Further Details =============== If UPLO = 'U', then A = U*D*U', where U = P(n)*U(n)* ... *P(k)U(k)* ..., i.e., U is a product of terms P(k)*U(k), where k decreases from n to 1 in steps of 1 or 2, and D is a block diagonal matrix with 1-by-1 and 2-by-2 diagonal blocks D(k). P(k) is a permutation matrix as defined by IPIV(k), and U(k) is a unit upper triangular matrix, such that if the diagonal block D(k) is of order s (s = 1 or 2), then ( I v 0 ) k-s U(k) = ( 0 I 0 ) s ( 0 0 I ) n-k k-s s n-k If s = 1, D(k) overwrites A(k,k), and v overwrites A(1:k-1,k). If s = 2, the upper triangle of D(k) overwrites A(k-1,k-1), A(k-1,k), and A(k,k), and v overwrites A(1:k-2,k-1:k). If UPLO = 'L', then A = L*D*L', where L = P(1)*L(1)* ... *P(k)*L(k)* ..., i.e., L is a product of terms P(k)*L(k), where k increases from 1 to n in steps of 1 or 2, and D is a block diagonal matrix with 1-by-1 and 2-by-2 diagonal blocks D(k). P(k) is a permutation matrix as defined by IPIV(k), and L(k) is a unit lower triangular matrix, such that if the diagonal block D(k) is of order s (s = 1 or 2), then ( I 0 0 ) k-1 L(k) = ( 0 I 0 ) s ( 0 v I ) n-k-s+1 k-1 s n-k-s+1 If s = 1, D(k) overwrites A(k,k), and v overwrites A(k+1:n,k). If s = 2, the lower triangle of D(k) overwrites A(k,k), A(k+1,k), and A(k+1,k+1), and v overwrites A(k+2:n,k:k+1). ===================================================================== Test the input parameters. Parameter adjustments Function Body */ /* Table of constant values */ static int c__1 = 1; /* System generated locals */ int i__1; double d__1, d__2, d__3; /* Builtin functions */ double sqrt(double); /* Local variables */ static int imax, jmax; extern /* Subroutine */ int drot_(int *, double *, int *, double *, int *, double *, double *), dspr_(char * , int *, double *, double *, int *, double *); static double c; static int j, k; static double s, t, alpha; extern /* Subroutine */ int dscal_(int *, double *, double *, int *); extern logical lsame_(char *, char *); extern /* Subroutine */ int dswap_(int *, double *, int *, double *, int *); static int kstep; static logical upper; static double r1, r2; extern /* Subroutine */ int dlaev2_(double *, double *, double *, double *, double *, double *, double *); static int kc, kk, kp; static double absakk; static int kx; extern int idamax_(int *, double *, int *); extern /* Subroutine */ int xerbla_(char *, int *); static double colmax, rowmax; static int knc, kpc, npp; #define IPIV(I) ipiv[(I)-1] #define AP(I) ap[(I)-1] *info = 0; upper = lsame_(uplo, "U"); if (! upper && ! lsame_(uplo, "L")) { *info = -1; } else if (*n < 0) { *info = -2; } if (*info != 0) { i__1 = -(*info); xerbla_("DSPTRF", &i__1); return 0; } /* Initialize ALPHA for use in choosing pivot block size. */ alpha = (sqrt(17.) + 1.) / 8.; if (upper) { /* Factorize A as U*D*U' using the upper triangle of A K is the main loop index, decreasing from N to 1 in steps of 1 or 2 */ k = *n; kc = (*n - 1) * *n / 2 + 1; L10: knc = kc; /* If K < 1, exit from loop */ if (k < 1) { goto L70; } kstep = 1; /* Determine rows and columns to be interchanged and whether a 1-by-1 or 2-by-2 pivot block will be used */ absakk = (d__1 = AP(kc + k - 1), abs(d__1)); /* IMAX is the row-index of the largest off-diagonal element in column K, and COLMAX is its absolute value */ if (k > 1) { i__1 = k - 1; imax = idamax_(&i__1, &AP(kc), &c__1); colmax = (d__1 = AP(kc + imax - 1), abs(d__1)); } else { colmax = 0.; } if (max(absakk,colmax) == 0.) { /* Column K is zero: set INFO and continue */ if (*info == 0) { *info = k; } kp = k; } else { if (absakk >= alpha * colmax) { /* no interchange, use 1-by-1 pivot block */ kp = k; } else { /* JMAX is the column-index of the largest off-di agonal element in row IMAX, and ROWMAX is its absolut e value */ rowmax = 0.; jmax = imax; kx = imax * (imax + 1) / 2 + imax; i__1 = k; for (j = imax + 1; j <= k; ++j) { if ((d__1 = AP(kx), abs(d__1)) > rowmax) { rowmax = (d__1 = AP(kx), abs(d__1)); jmax = j; } kx += j; /* L20: */ } kpc = (imax - 1) * imax / 2 + 1; if (imax > 1) { i__1 = imax - 1; jmax = idamax_(&i__1, &AP(kpc), &c__1); /* Computing MAX */ d__2 = rowmax, d__3 = (d__1 = AP(kpc + jmax - 1), abs( d__1)); rowmax = max(d__2,d__3); } if (absakk >= alpha * colmax * (colmax / rowmax)) { /* no interchange, use 1-by-1 pivot block */ kp = k; } else if ((d__1 = AP(kpc + imax - 1), abs(d__1)) >= alpha * rowmax) { /* interchange rows and columns K and IMAX , use 1-by-1 pivot block */ kp = imax; } else { /* interchange rows and columns K-1 and IM AX, use 2-by-2 pivot block */ kp = imax; kstep = 2; } } kk = k - kstep + 1; if (kstep == 2) { knc = knc - k + 1; } if (kp != kk) { /* Interchange rows and columns KK and KP in the leading submatrix A(1:k,1:k) */ i__1 = kp - 1; dswap_(&i__1, &AP(knc), &c__1, &AP(kpc), &c__1); kx = kpc + kp - 1; i__1 = kk - 1; for (j = kp + 1; j <= kk-1; ++j) { kx = kx + j - 1; t = AP(knc + j - 1); AP(knc + j - 1) = AP(kx); AP(kx) = t; /* L30: */ } t = AP(knc + kk - 1); AP(knc + kk - 1) = AP(kpc + kp - 1); AP(kpc + kp - 1) = t; if (kstep == 2) { t = AP(kc + k - 2); AP(kc + k - 2) = AP(kc + kp - 1); AP(kc + kp - 1) = t; } } /* Update the leading submatrix */ if (kstep == 1) { /* 1-by-1 pivot block D(k): column k now holds W(k) = U(k)*D(k) where U(k) is the k-th column of U Perform a rank-1 update of A(1:k-1,1:k-1) as A := A - U(k)*D(k)*U(k)' = A - W(k)*1/D(k)*W(k )' */ r1 = 1. / AP(kc + k - 1); i__1 = k - 1; d__1 = -r1; dspr_(uplo, &i__1, &d__1, &AP(kc), &c__1, &AP(1)); /* Store U(k) in column k */ i__1 = k - 1; dscal_(&i__1, &r1, &AP(kc), &c__1); } else { /* 2-by-2 pivot block D(k): columns k and k-1 now hold ( W(k-1) W(k) ) = ( U(k-1) U(k) )*D(k) where U(k) and U(k-1) are the k-th and (k-1)-t h columns of U Perform a rank-2 update of A(1:k-2,1:k-2) as A := A - ( U(k-1) U(k) )*D(k)*( U(k-1) U(k) )' = A - ( W(k-1) W(k) )*inv(D(k))*( W(k-1) W( k) )' Convert this to two rank-1 updates by using th e eigen- decomposition of D(k) */ dlaev2_(&AP(kc - 1), &AP(kc + k - 2), &AP(kc + k - 1), &r1, & r2, &c, &s); r1 = 1. / r1; r2 = 1. / r2; i__1 = k - 2; drot_(&i__1, &AP(knc), &c__1, &AP(kc), &c__1, &c, &s); i__1 = k - 2; d__1 = -r1; dspr_(uplo, &i__1, &d__1, &AP(knc), &c__1, &AP(1)); i__1 = k - 2; d__1 = -r2; dspr_(uplo, &i__1, &d__1, &AP(kc), &c__1, &AP(1)); /* Store U(k) and U(k-1) in columns k and k-1 */ i__1 = k - 2; dscal_(&i__1, &r1, &AP(knc), &c__1); i__1 = k - 2; dscal_(&i__1, &r2, &AP(kc), &c__1); i__1 = k - 2; d__1 = -s; drot_(&i__1, &AP(knc), &c__1, &AP(kc), &c__1, &c, &d__1); } } /* Store details of the interchanges in IPIV */ if (kstep == 1) { IPIV(k) = kp; } else { IPIV(k) = -kp; IPIV(k - 1) = -kp; } /* Decrease K and return to the start of the main loop */ k -= kstep; kc = knc - k; goto L10; } else { /* Factorize A as L*D*L' using the lower triangle of A K is the main loop index, increasing from 1 to N in steps of 1 or 2 */ k = 1; kc = 1; npp = *n * (*n + 1) / 2; L40: knc = kc; /* If K > N, exit from loop */ if (k > *n) { goto L70; } kstep = 1; /* Determine rows and columns to be interchanged and whether a 1-by-1 or 2-by-2 pivot block will be used */ absakk = (d__1 = AP(kc), abs(d__1)); /* IMAX is the row-index of the largest off-diagonal element in column K, and COLMAX is its absolute value */ if (k < *n) { i__1 = *n - k; imax = k + idamax_(&i__1, &AP(kc + 1), &c__1); colmax = (d__1 = AP(kc + imax - k), abs(d__1)); } else { colmax = 0.; } if (max(absakk,colmax) == 0.) { /* Column K is zero: set INFO and continue */ if (*info == 0) { *info = k; } kp = k; } else { if (absakk >= alpha * colmax) { /* no interchange, use 1-by-1 pivot block */ kp = k; } else { /* JMAX is the column-index of the largest off-di agonal element in row IMAX, and ROWMAX is its absolut e value */ rowmax = 0.; kx = kc + imax - k; i__1 = imax - 1; for (j = k; j <= imax-1; ++j) { if ((d__1 = AP(kx), abs(d__1)) > rowmax) { rowmax = (d__1 = AP(kx), abs(d__1)); jmax = j; } kx = kx + *n - j; /* L50: */ } kpc = npp - (*n - imax + 1) * (*n - imax + 2) / 2 + 1; if (imax < *n) { i__1 = *n - imax; jmax = imax + idamax_(&i__1, &AP(kpc + 1), &c__1); /* Computing MAX */ d__2 = rowmax, d__3 = (d__1 = AP(kpc + jmax - imax), abs( d__1)); rowmax = max(d__2,d__3); } if (absakk >= alpha * colmax * (colmax / rowmax)) { /* no interchange, use 1-by-1 pivot block */ kp = k; } else if ((d__1 = AP(kpc), abs(d__1)) >= alpha * rowmax) { /* interchange rows and columns K and IMAX , use 1-by-1 pivot block */ kp = imax; } else { /* interchange rows and columns K+1 and IM AX, use 2-by-2 pivot block */ kp = imax; kstep = 2; } } kk = k + kstep - 1; if (kstep == 2) { knc = knc + *n - k + 1; } if (kp != kk) { /* Interchange rows and columns KK and KP in the trailing submatrix A(k:n,k:n) */ if (kp < *n) { i__1 = *n - kp; dswap_(&i__1, &AP(knc + kp - kk + 1), &c__1, &AP(kpc + 1), &c__1); } kx = knc + kp - kk; i__1 = kp - 1; for (j = kk + 1; j <= kp-1; ++j) { kx = kx + *n - j + 1; t = AP(knc + j - kk); AP(knc + j - kk) = AP(kx); AP(kx) = t; /* L60: */ } t = AP(knc); AP(knc) = AP(kpc); AP(kpc) = t; if (kstep == 2) { t = AP(kc + 1); AP(kc + 1) = AP(kc + kp - k); AP(kc + kp - k) = t; } } /* Update the trailing submatrix */ if (kstep == 1) { /* 1-by-1 pivot block D(k): column k now holds W(k) = L(k)*D(k) where L(k) is the k-th column of L */ if (k < *n) { /* Perform a rank-1 update of A(k+1:n,k+1: n) as A := A - L(k)*D(k)*L(k)' = A - W(k)*(1/ D(k))*W(k)' */ r1 = 1. / AP(kc); i__1 = *n - k; d__1 = -r1; dspr_(uplo, &i__1, &d__1, &AP(kc + 1), &c__1, &AP(kc + *n - k + 1)); /* Store L(k) in column K */ i__1 = *n - k; dscal_(&i__1, &r1, &AP(kc + 1), &c__1); } } else { /* 2-by-2 pivot block D(k): columns K and K+1 now hold ( W(k) W(k+1) ) = ( L(k) L(k+1) )*D(k) where L(k) and L(k+1) are the k-th and (k+1)-t h columns of L */ if (k < *n - 1) { /* Perform a rank-2 update of A(k+2:n,k+2: n) as A := A - ( L(k) L(k+1) )*D(k)*( L(k) L( k+1) )' = A - ( W(k) W(k+1) )*inv(D(k))*( W( k) W(k+1) )' Convert this to two rank-1 updates by u sing the eigen- decomposition of D(k) */ dlaev2_(&AP(kc), &AP(kc + 1), &AP(knc), &r1, &r2, &c, &s); r1 = 1. / r1; r2 = 1. / r2; i__1 = *n - k - 1; drot_(&i__1, &AP(kc + 2), &c__1, &AP(knc + 1), &c__1, &c, &s); i__1 = *n - k - 1; d__1 = -r1; dspr_(uplo, &i__1, &d__1, &AP(kc + 2), &c__1, &AP(knc + * n - k)); i__1 = *n - k - 1; d__1 = -r2; dspr_(uplo, &i__1, &d__1, &AP(knc + 1), &c__1, &AP(knc + * n - k)); /* Store L(k) and L(k+1) in columns k and k+1 */ i__1 = *n - k - 1; dscal_(&i__1, &r1, &AP(kc + 2), &c__1); i__1 = *n - k - 1; dscal_(&i__1, &r2, &AP(knc + 1), &c__1); i__1 = *n - k - 1; d__1 = -s; drot_(&i__1, &AP(kc + 2), &c__1, &AP(knc + 1), &c__1, &c, &d__1); } } } /* Store details of the interchanges in IPIV */ if (kstep == 1) { IPIV(k) = kp; } else { IPIV(k) = -kp; IPIV(k + 1) = -kp; } /* Increase K and return to the start of the main loop */ k += kstep; kc = knc + *n - k + 2; goto L40; } L70: return 0; /* End of DSPTRF */ } /* dsptrf_ */ int dsptri_(char *uplo, int *n, double *ap, int * ipiv, double *work, int *info) { /* -- LAPACK routine (version 2.0) -- Univ. of Tennessee, Univ. of California Berkeley, NAG Ltd., Courant Institute, Argonne National Lab, and Rice University March 31, 1993 Purpose ======= DSPTRI computes the inverse of a double symmetric indefinite matrix A in packed storage using the factorization A = U*D*U**T or A = L*D*L**T computed by DSPTRF. Arguments ========= UPLO (input) CHARACTER*1 Specifies whether the details of the factorization are stored as an upper or lower triangular matrix. = 'U': Upper triangular, form is A = U*D*U**T; = 'L': Lower triangular, form is A = L*D*L**T. N (input) INT The order of the matrix A. N >= 0. AP (input/output) DOUBLE PRECISION array, dimension (N*(N+1)/2) On entry, the block diagonal matrix D and the multipliers used to obtain the factor U or L as computed by DSPTRF, stored as a packed triangular matrix. On exit, if INFO = 0, the (symmetric) inverse of the original matrix, stored as a packed triangular matrix. The j-th column of inv(A) is stored in the array AP as follows: if UPLO = 'U', AP(i + (j-1)*j/2) = inv(A)(i,j) for 1<=i<=j; if UPLO = 'L', AP(i + (j-1)*(2n-j)/2) = inv(A)(i,j) for j<=i<=n. IPIV (input) INT array, dimension (N) Details of the interchanges and the block structure of D as determined by DSPTRF. WORK (workspace) DOUBLE PRECISION array, dimension (N) INFO (output) INT = 0: successful exit < 0: if INFO = -i, the i-th argument had an illegal value > 0: if INFO = i, D(i,i) = 0; the matrix is singular and its inverse could not be computed. ===================================================================== Test the input parameters. Parameter adjustments Function Body */ /* Table of constant values */ static int c__1 = 1; static double c_b11 = -1.; static double c_b13 = 0.; /* System generated locals */ int i__1; double d__1; /* Local variables */ extern double ddot_(int *, double *, int *, double *, int *); static double temp, akkp1, d; static int j, k; static double t; extern logical lsame_(char *, char *); extern /* Subroutine */ int dcopy_(int *, double *, int *, double *, int *), dswap_(int *, double *, int *, double *, int *); static int kstep; extern /* Subroutine */ int dspmv_(char *, int *, double *, double *, double *, int *, double *, double *, int *); static logical upper; static double ak; static int kc, kp, kx; extern /* Subroutine */ int xerbla_(char *, int *); static int kcnext, kpc, npp; static double akp1; #define WORK(I) work[(I)-1] #define IPIV(I) ipiv[(I)-1] #define AP(I) ap[(I)-1] *info = 0; upper = lsame_(uplo, "U"); if (! upper && ! lsame_(uplo, "L")) { *info = -1; } else if (*n < 0) { *info = -2; } if (*info != 0) { i__1 = -(*info); xerbla_("DSPTRI", &i__1); return 0; } /* Quick return if possible */ if (*n == 0) { return 0; } /* Check that the diagonal matrix D is nonsingular. */ if (upper) { /* Upper triangular storage: examine D from bottom to top */ kp = *n * (*n + 1) / 2; for (*info = *n; *info >= 1; --(*info)) { if (IPIV(*info) > 0 && AP(kp) == 0.) { return 0; } kp -= *info; /* L10: */ } } else { /* Lower triangular storage: examine D from top to bottom. */ kp = 1; i__1 = *n; for (*info = 1; *info <= i__1; ++(*info)) { if (IPIV(*info) > 0 && AP(kp) == 0.) { return 0; } kp = kp + *n - *info + 1; /* L20: */ } } *info = 0; if (upper) { /* Compute inv(A) from the factorization A = U*D*U'. K is the main loop index, increasing from 1 to N in steps of 1 or 2, depending on the size of the diagonal blocks. */ k = 1; kc = 1; L30: /* If K > N, exit from loop. */ if (k > *n) { goto L50; } kcnext = kc + k; if (IPIV(k) > 0) { /* 1 x 1 diagonal block Invert the diagonal block. */ AP(kc + k - 1) = 1. / AP(kc + k - 1); /* Compute column K of the inverse. */ if (k > 1) { i__1 = k - 1; dcopy_(&i__1, &AP(kc), &c__1, &WORK(1), &c__1); i__1 = k - 1; dspmv_(uplo, &i__1, &c_b11, &AP(1), &WORK(1), &c__1, &c_b13, & AP(kc), &c__1); i__1 = k - 1; AP(kc + k - 1) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kc), & c__1); } kstep = 1; } else { /* 2 x 2 diagonal block Invert the diagonal block. */ t = (d__1 = AP(kcnext + k - 1), abs(d__1)); ak = AP(kc + k - 1) / t; akp1 = AP(kcnext + k) / t; akkp1 = AP(kcnext + k - 1) / t; d = t * (ak * akp1 - 1.); AP(kc + k - 1) = akp1 / d; AP(kcnext + k) = ak / d; AP(kcnext + k - 1) = -akkp1 / d; /* Compute columns K and K+1 of the inverse. */ if (k > 1) { i__1 = k - 1; dcopy_(&i__1, &AP(kc), &c__1, &WORK(1), &c__1); i__1 = k - 1; dspmv_(uplo, &i__1, &c_b11, &AP(1), &WORK(1), &c__1, &c_b13, & AP(kc), &c__1); i__1 = k - 1; AP(kc + k - 1) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kc), & c__1); i__1 = k - 1; AP(kcnext + k - 1) -= ddot_(&i__1, &AP(kc), &c__1, &AP(kcnext) , &c__1); i__1 = k - 1; dcopy_(&i__1, &AP(kcnext), &c__1, &WORK(1), &c__1); i__1 = k - 1; dspmv_(uplo, &i__1, &c_b11, &AP(1), &WORK(1), &c__1, &c_b13, & AP(kcnext), &c__1); i__1 = k - 1; AP(kcnext + k) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kcnext), & c__1); } kstep = 2; kcnext = kcnext + k + 1; } kp = (i__1 = IPIV(k), abs(i__1)); if (kp != k) { /* Interchange rows and columns K and KP in the leading submatrix A(1:k+1,1:k+1) */ kpc = (kp - 1) * kp / 2 + 1; i__1 = kp - 1; dswap_(&i__1, &AP(kc), &c__1, &AP(kpc), &c__1); kx = kpc + kp - 1; i__1 = k - 1; for (j = kp + 1; j <= k-1; ++j) { kx = kx + j - 1; temp = AP(kc + j - 1); AP(kc + j - 1) = AP(kx); AP(kx) = temp; /* L40: */ } temp = AP(kc + k - 1); AP(kc + k - 1) = AP(kpc + kp - 1); AP(kpc + kp - 1) = temp; if (kstep == 2) { temp = AP(kc + k + k - 1); AP(kc + k + k - 1) = AP(kc + k + kp - 1); AP(kc + k + kp - 1) = temp; } } k += kstep; kc = kcnext; goto L30; L50: ; } else { /* Compute inv(A) from the factorization A = L*D*L'. K is the main loop index, increasing from 1 to N in steps of 1 or 2, depending on the size of the diagonal blocks. */ npp = *n * (*n + 1) / 2; k = *n; kc = npp; L60: /* If K < 1, exit from loop. */ if (k < 1) { goto L80; } kcnext = kc - (*n - k + 2); if (IPIV(k) > 0) { /* 1 x 1 diagonal block Invert the diagonal block. */ AP(kc) = 1. / AP(kc); /* Compute column K of the inverse. */ if (k < *n) { i__1 = *n - k; dcopy_(&i__1, &AP(kc + 1), &c__1, &WORK(1), &c__1); i__1 = *n - k; dspmv_(uplo, &i__1, &c_b11, &AP(kc + *n - k + 1), &WORK(1), & c__1, &c_b13, &AP(kc + 1), &c__1); i__1 = *n - k; AP(kc) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kc + 1), &c__1); } kstep = 1; } else { /* 2 x 2 diagonal block Invert the diagonal block. */ t = (d__1 = AP(kcnext + 1), abs(d__1)); ak = AP(kcnext) / t; akp1 = AP(kc) / t; akkp1 = AP(kcnext + 1) / t; d = t * (ak * akp1 - 1.); AP(kcnext) = akp1 / d; AP(kc) = ak / d; AP(kcnext + 1) = -akkp1 / d; /* Compute columns K-1 and K of the inverse. */ if (k < *n) { i__1 = *n - k; dcopy_(&i__1, &AP(kc + 1), &c__1, &WORK(1), &c__1); i__1 = *n - k; dspmv_(uplo, &i__1, &c_b11, &AP(kc + (*n - k + 1)), &WORK(1), &c__1, &c_b13, &AP(kc + 1), &c__1); i__1 = *n - k; AP(kc) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kc + 1), &c__1); i__1 = *n - k; AP(kcnext + 1) -= ddot_(&i__1, &AP(kc + 1), &c__1, &AP(kcnext + 2), &c__1); i__1 = *n - k; dcopy_(&i__1, &AP(kcnext + 2), &c__1, &WORK(1), &c__1); i__1 = *n - k; dspmv_(uplo, &i__1, &c_b11, &AP(kc + (*n - k + 1)), &WORK(1), &c__1, &c_b13, &AP(kcnext + 2), &c__1); i__1 = *n - k; AP(kcnext) -= ddot_(&i__1, &WORK(1), &c__1, &AP(kcnext + 2), & c__1); } kstep = 2; kcnext -= *n - k + 3; } kp = (i__1 = IPIV(k), abs(i__1)); if (kp != k) { /* Interchange rows and columns K and KP in the trailing submatrix A(k-1:n,k-1:n) */ kpc = npp - (*n - kp + 1) * (*n - kp + 2) / 2 + 1; if (kp < *n) { i__1 = *n - kp; dswap_(&i__1, &AP(kc + kp - k + 1), &c__1, &AP(kpc + 1), & c__1); } kx = kc + kp - k; i__1 = kp - 1; for (j = k + 1; j <= kp-1; ++j) { kx = kx + *n - j + 1; temp = AP(kc + j - k); AP(kc + j - k) = AP(kx); AP(kx) = temp; /* L70: */ } temp = AP(kc); AP(kc) = AP(kpc); AP(kpc) = temp; if (kstep == 2) { temp = AP(kc - *n + k - 1); AP(kc - *n + k - 1) = AP(kc - *n + kp - 1); AP(kc - *n + kp - 1) = temp; } } k -= kstep; kc = kcnext; goto L60; L80: ; } return 0; /* End of DSPTRI */ } /* dsptri_ */ #undef calloc SHAR_EOF fi if test -f 'predict.polymars' then echo shar: "will not over-write existing file 'predict.polymars'" else cat << \SHAR_EOF > 'predict.polymars' .BG .FN predict.polymars .TL predict.polymars - fitted values from a `polymars' model .DN produces fitted values for a model of class polymars. .CS predict.polymars(polymars.model, x, classify=F, intercept) .RA .AG polymars.model an object of the class `polymars', usually the result of a call of the function polymars. .AG x the predictor values at which the fitted values will be computed. The predictor values can be in a number of formats. It can take the form of a vector of length equal to the number of predictors in the original data set or it can be shortened to the length of only those predictors that occur in the model, in the same order as they appear in the original data set. Similarly, x can take the form of a matrix with the number of columns equal to the number of predictors in the original data set, or shortened to the number of predictors in the model. .OA .AG classify if the original call to polymars was for a classification problem and you would like the classifications for the x-values, set this option = T. Otherwise the function returns a response column for each class (the highest values in each row is its class for the case when classify = T). .AG intercept is by default = T so the models intercept coefficient is included in the evaluation. Setting intercept = F evaluates without the intercept. The intercept may also be given any numerical value which overrides the fitted coefficient from the model. .RT the returned object is a matrix of fitted values, fitted according to the model produced by the function polymars. The number of columns in the returned matrix equals the number of responses in the original call to the polymars function. .DT .SA polymars, plot.polymars, summary.polymars .EX state.pm<-polymars(state.region,state.x77,knots=15,classify=T,additive=T) predict(state.pm,x=state.x77,classify=T) .WR SHAR_EOF fi if test -f 'print.polymars' then echo shar: "will not over-write existing file 'print.polymars'" else cat << \SHAR_EOF > 'print.polymars' .BG .FN print.polymars .TL print.polymars - summarizes a polymars model .DN Gives details of a Multi-response Multivariate Adaptive Regression Splines fit .CS print.polymars(mars.model) .RA .AG mars.model a model returned from a call to polymars .RT The returned object contains information about the fitting steps and the model selected. The first data frame contains a row for each step of the fitting procedure. In the columns are: a 1 for an addition step or a 0 for a deletion step, the size of the model at each step, residual sums of squares (RSS) and the generalized cross validation value (GCV), testset residual sums of squares or testset misclassification, whatever was used for the model selection. The second data frame, model, contains a row for each basis function of the model. Each row corresponds to one basis function (with two possible components). The pred1 column contains the indices of the first predictor of the basis function. Column knot1 is a possible knot in this predictor. If this column is NA, the first component is linear. If any of the basis functions of the model is categorical then there will be a level1 column. Column pred2 is the possible second predictor involved (if it is NA the basis function only depends on one predictor). Column knot2 contains the possible knot for the predictor pred2, and it is NA when this component is linear. This is a similar format to the startmodel argument together with an additional first row corresponding to the intercept but the startmodel doesn't use a separate column to specify levels of a categorical variable . If any predictor in pred2 is categorical then there will be a level2 column. The column "coefs" (more than one column in the case of multiple response regression) contains the coefficients. .DT print.polymars defaults to summary.polymars. .SA plot.polymars, polymars.fun, summary.polymars, print.polymars .WR SHAR_EOF fi if test -f 'summary.polymars' then echo shar: "will not over-write existing file 'summary.polymars'" else cat << \SHAR_EOF > 'summary.polymars' .BG .FN summary.polymars .TL summary.polymars - summarizes a polymars model .DN Gives details of a Multi-response Multivariate Adaptive Regression Splines fit .CS summary.polymars(mars.model) .RA .AG mars.model a model returned from a call to polymars .RT The returned object contains information about the fitting steps and the model selected. The first data frame contains a row for each step of the fitting procedure. In the columns are: a 1 for an addition step or a 0 for a deletion step, the size of the model at each step, residual sums of squares (RSS) and the generalized cross validation value (GCV), testset residual sums of squares or testset misclassification, whatever was used for the model selection. The second data frame, model, contains a row for each basis function of the model. Each row corresponds to one basis function (with two possible components). The pred1 column contains the indices of the first predictor of the basis function. Column knot1 is a possible knot in this predictor. If this column is NA, the first component is linear. If any of the basis functions of the model is categorical then there will be a level1 column. Column pred2 is the possible second predictor involved (if it is NA the basis function only depends on one predictor). Column knot2 contains the possible knot for the predictor pred2, and it is NA when this component is linear. This is a similar format to the startmodel argument together with an additional first row corresponding to the intercept but the startmodel doesn't use a separate column to specify levels of a categorical variable . If any predictor in pred2 is categorical then there will be a level2 column. The column "coefs" (more than one column in the case of multiple response regression) contains the coefficients. .DT is.polymars and summary.polymars are available. print.polymars defaults to summary.polymars. .SA plot.polymars, polymars.fun, summary.polymars, print.polymars .EX fit <- polymars(mydata[,1], mydata[,c(2,3,4,5,6)]) summary(fit) .WR SHAR_EOF fi exit 0 # End of shell archive