Results, documents and R

Methods text

Model › Text for the Methods section: a paragraph written from the model, in French or English (data, transformations, scaling, spectral preprocessing, method, cross-validation, R², Q², CV-ANOVA, classification, software and R version), with its references. "Copy the text" button. Read it over and complete it (experimental design, sampling...) before publishing.

Model report

File › Model report (or right-click on the model): a PDF or Word (.docx) document with a title, a comment and the parts ticked (summary, scores, loadings, DModX, VIP, coefficients, CV-ANOVA, classification, list of variables...). Plots are formatted as in their open panels. The document opens once written.

Documents (articles, reports)

File › New document (or right-click a project in the explorer): a text kept in the project (an article, a report, notes), written in Markdown on the left, with its preview on the right, drawn again as you type. It is saved with the project and in its recovery copies; Edit › Undo leaves it alone, and Ctrl+Z in the text undoes the text.

Start from a blank document, an article (introduction, methods, results, discussion) or an analysis report (for each model: summary, plots, methods), the models ticked already cited.

The project's results are cited between double braces; the preview and the compiled file show their values:

{{M1.Q2}}  {{M1.R2Y}}  {{M1.R2X:%}} (a percentage)  {{M1.Q2:3}} (3 decimals)  {{M1.N}}
{{M1.gain}}  {{M1.vip.sugar}}  {{M1.coef.sugar}} (scaled coefficient and its 95 % interval)
{{M5.coef.year}} (a regression: estimate, interval and p)  {{M5.p.year}}  {{M1.cvanova.p}}
{{M1.scores | caption}} (a plot, as in its window)  {{M5.effets.year}} (one term's plot)
{{M1.table.resume | caption}} (a table of the model report)  {{M1.methodes}} (Methods text)
{{M1.resultats}} (Results text: the fit, each coefficient with its interval and p, as a factor
or an odds ratio for a log or logit link, the significant terms, curves, random effects; for a
PCA, PLS or OPLS, the variance explained, Q², CV-ANOVA and the most important variables)

A plot, a table, the Methods text or the Results text goes alone on its line. Model › Text for the Methods section shows the Results text under the Methods. Plots and tables are numbered in order (Figure 1, Table 1); with a label, {{M1.scores #fig-scores | caption}}, {{ref:fig-scores}} writes "Figure 1" anywhere in the text. Type {{ to list the project's models, then what the one chosen offers, with its current value; Insert, in the toolbar, does the same from menus, and a model or its view in the explorer has "Insert in the document" (right-click).

A reference that can't be resolved (a model that doesn't exist, a variable it doesn't have) is in red, with the reason when you hover it. A model of the project not computed yet is computed when the document cites it, without opening its views. Numbers follow the document's language (0,65 in French, 0.65 in English), as do "Figure", "Table" and the Methods text.

Compile: a Word file (.docx: headings, lists, tables, plots and captions in Word's styles), a PDF, an HTML page (a single file, plots included), or a Quarto file (.qmd). When Quarto is installed (quarto.org, or with RStudio), "Compile with Quarto" makes Word, PDF (Typst) or HTML from it, with what Quarto adds: a bibliography (bibliography: refs.bib in the header, [@key] in the text), cross-references, journal templates. A YAML header (title, author, date) at the top of the document is read; a line {{< pagebreak >}} starts a new page.

For Quarto, Compile's dialog asks for the results:

• written in: the values, plots (in a folder next to the .qmd) and tables as Nuée shows them;
• reproducible: R recomputes them from the raw data when Quarto renders the document. Next to the .qmd, the folder "<name>_R" holds each cited model's Export to R (File › Export to R: the script, its data as CSV) and nuee.R. The .qmd's first chunk runs those scripts again, each in an environment of its own; each cited value becomes an inline R expression, `r nuee_nombre(M1$components$Q2cum[2])`, that reads the recomputed value and writes it as Nuée does (0.65, 71%, p < 0.001); a plot becomes an R chunk that draws it with the script's code (in base R), a table a knitr::kable chunk of the script's objects. Correcting a value in the CSV, then rendering again, recomputes all that R writes. What the scripts don't have stays written in, and the .qmd's first chunk says so in a comment: a plot they don't draw (its image), the coefficient of an OPLS, or of a PLS validated with the full model's centres and weights (its script hasn't its interval), the Methods and Results texts (written by Nuée, not recomputed), a model whose type has no export to R yet (the dialog says so first). Before writing the .qmd, Nuée runs the scripts again and checks that each expression gives back the value it shows; any one that differs stays written in. R is needed, with its knitr and rmarkdown packages: Nuée has Quarto render with its own R when it has them (otherwise Quarto takes the computer's R, or QUARTO_R's). In the browser, the .qmd and what goes beside it come in a .zip.

Citations are written as in Quarto: [@wold2001], [see @wold2001, p. 33; @eriksson2006], @wold2001 in the sentence ("Wold et al. (2001)"), [-@wold2001] without the author. Typing @ lists the document's references, then Zotero's. The list of references comes at the end (under the last heading when it is "References"), or where the document puts {{bibliographie}}; author–year or numbered (Cite › Citation style), in the document's language. Where the references come from (the Cite button):

• BibTeX or CSL-JSON files, exported by Zotero, Mendeley, JabRef...: Cite › Link a file names one in the header (bibliography: refs.bib, relative to the project's folder, as Quarto reads it). Better BibTeX's automatic export keeps it up to date: a key not found has the file read again;
• Zotero on this computer (Zotero 7 or later, with Settings › Advanced › "Allow other applications on this computer to communicate with Zotero" ticked): its references are searched as you type @, or in Cite › Insert a citation, in My Library or in a group library (those Zotero syncs, under My Library in its window), whole or one of its collections; the library chosen in Insert a citation is kept, and typing @ searches it. Their keys are Zotero's own (Better BibTeX's when it is installed). With the Better BibTeX plugin, Cite › Pick in Zotero opens Zotero's own picker, in all its libraries, the page given there included. Nuée only reads the libraries. When Zotero doesn't list its group libraries, Insert a citation says why and offers My Library alone;
• BibTeX pasted (Cite › Paste BibTeX references).

A key is unique within a Zotero library, not across libraries: two different works can have the same key in My Library and in a group (bechet2019). A reference from Zotero whose key the document already gives to another work (from another library, or from a linked file) is cited with a letter added, @bechet2019a, and the status bar says so; the same item cited again, or the same work copied into another library (the same DOI, or the same title, year and first author), keeps the key the document has for it, whichever key Zotero gives it there. A file linked afterwards that gives the key of a reference kept from Zotero to another work: the file's is the one cited, and the status bar says so. The document keeps a copy of the references it cites, with the library and item each one from Zotero came from: sent to someone without the .bib, it still cites them (Cite › The document's references lists them, and removes those no longer cited). Compiled for Quarto, the citations stay as they are, the references cited in a .bib beside the .qmd, so that Quarto writes them in any journal's style (csl: style.csl in the header; the file is copied beside the .qmd).

A reference's language (Zotero's Language field: fr, de...) goes with it: styles that put English titles in title case, as Quarto's default one does, then leave a French title as it is written. Each document opens in a tab of its own. Plots in documents and model reports are drawn in the light theme, whatever the window's.

Spelling, in French and in English (and in the system's language): misspelled words are underlined; right-click one for suggestions (in every text field of the application).

Exporting to other software

File › Export data: the project's table (observation names, identifiers, variables). File › Export model results (Ctrl+E): the model's tables (observations: scores, T², DModX, fitted values; variables: loadings, VIP, intervals; components; coefficients; Y variables). Formats: Excel (one sheet per table), CSV, tab-separated text, R (.rds: a list of tables; .RData), SPSS (.sav), Stata (.dta), SAS transport (.xpt), jamovi (.omv), dBase (.dbf) and Weka (.arff); for the last ones, one file per table ("M1.sav", "M1 - variables.sav"...). Qualitative variables stay qualitative (R factors, SPSS and Stata value labels, jamovi nominal variables). For SPSS, Stata, SAS and dBase, variable names are made valid (unaccented letters, digits, "_"); the original name becomes the variable label. The variables' units and descriptions (Edit › Units and descriptions) go into a "Variables" sheet of an Excel workbook, read back when Nuée opens it, a "Variables" table of a GeoPackage, the descriptions of a jamovi file and the attributes of an R table (see Cleaning the data).

SAS: transport format version 5, which SAS reads with libname xport (names of 8 characters, labels of 40, text of 200 at most). dBase: text in Windows-1252; in LibreOffice, choose the "Western Europe (Windows-1252)" character set when opening. The files written were opened in jamovi, PSPP (for SPSS), Weka, LibreOffice and pandas.

Export to R (script, R Markdown, Quarto)

File › Export to R: a file that refits the model without Nuée, with R alone, to send to a colleague or attach to a paper. The calculation is written for that model alone, in plain R with no package (a regression: lm, or nlme and mgcv, which come with R; an NMDS: MASS, which comes with R too; TRIM and distance sampling, and boosted trees with rpart: the functions of Nuée's engine, copied into the script; a random forest, the one exception: the ranger package, which Nuée installs with itself), commented in the export's language, in four parts:

1. the functions: the NIPALS algorithm, the cross-validation, and only what this model uses (missing values, spectral preprocessing, OPLS...);
2. the data, in the script or in a CSV file next to it ("M1 - donnees.csv"), numbers written without loss and categories in their order;
3. the model: its X and Y terms (transformations, categories, cross terms), their scaling, the fit, then R2 and Q2, T2 and DModX with their limits, VIP, coefficients (a PLS's with their 95% jackknife interval), RMSEE and RMSEcv, CV-ANOVA, classification for a PLS-DA, test set;
4. the results: the tables, the permutation test (commented out) and the plots (scores, loadings, VIP, observed / fitted, T2 and DModX).

Format: R script (.R); or an R Markdown (.Rmd) or Quarto (.qmd) report, which adds the Methods text, tables and plots (scores, loadings, VIP, observed / predicted, T2, DModX) and the references; it compiles in RStudio (Knit button) or with quarto render, and needs the knitr package. Extra files, as you choose: renv.lock (versions of R and packages, for renv::restore()), the data and fitted model as .RData (load() gives them back without refitting), an RStudio project (.Rproj, best in a folder of its own).

Models: this one, or all the models of the session (the latest fit of each) in one .R script: the functions and the data once, then a block per model under its name (M1, M2...).

Before writing, the application refits the script's models and checks they give the same results; otherwise it says so.

R console

View › R console (Ctrl+J): a panel below the tabs, with

• the log: for each action of the application, the call it made to the engine, written in R (P1 <- lire_projet(...), d <- modele_du_projet(P1, 5), M4 <- ajuster(P1, d)...), to copy into a script (Copy button), and for each fitted model what R prints of it (components, Y, limits, variables, observations; print(M4) does the same); the "Application actions" box hides or shows them;

• a prompt to type R, run in the application's R session. m is the model shown, p its project; P1, M4... the open projects and models, by their names in the log. For example: m$statistiques$variables, summary(p$donnees), plot(m$T[, 1], m$T[, 2]) (the plot shows in the console), models(), projects(), definition("M4").

  Enter runs the command, Shift+Enter starts a new line, ↑ and ↓ recall commands.

  Tab completes the name begun: objects, m$... and p$... (the names they hold), the arguments of the function whose parenthesis is open; when several fit, a list opens (arrows, then Tab or Enter). The application's shortcuts (Ctrl+S, Ctrl+K...) also work while typing.

Changing a project in the console (p$donnees[...] <- ...) changes it in the application. Correcting the engine: engine_folder() gives the folder of the application's R files; after a correction, reload_engine() reads them again, then refit the model. Updating Nuée replaces these files.

A long command blocks the application until the computation ends.

Checking the engine: check_engine() refits the imported models of the open projects and compares them with the values stored in their files (R2X, R2Y, Q2), validated like the files; check_engine("folder") does the same for the .usp and .json files of a folder. To do after a correction of the engine. Q2 counts in the verdict, except a PCA's, only close (see "What differs"). The French names of these helpers (modeles(), projets(), dossier_moteur(), recharger_moteur(), verifier_moteur()) work too.

The code

Every computation is in resources\moteur\R (R, commented in French):

pretraitement.R        transformations, centring, scaling
nipals.R               PCA and PLS (NIPALS algorithm, missing data)
validation_croisee.R   cross-validation (three ways), Q2
hierarchique.R         hierarchical models: honest Q2 (base models refitted)
regression.R           regression models (lm, nlme, mgcv), AIC, block contributions
statistiques.R         T2, DModX, VIP, coefficients, intervals, limits
validation.R           permutations, CV-ANOVA, PLS-DA classification and ROC curves
opls.R                 OPLS and OPLS-DA
diagnostic.R           contributions, data checks, variable statistics
variables.R            scores as variables, classes, edited values, computed variables
projet.R               reading projects, file history
verification.R         comparison with the results stored in .usp files
script.R               export of a model as a standalone R script or report
methodes.R             Methods text and references
comparaisons.R         statistical tests
modeles_classe.R       class models (one PCA per class), Coomans plot
formats.R              reading and writing tables (Excel, text, R, SPSS, Stata, SAS, jamovi...)
spectres.R             spectral preprocessing (SNV, MSC, detrend, baseline, Savitzky–Golay)
fichiers_spectres.R    reading spectra (JCAMP-DX, SPC, text), joined information
jeu_test.R             test set: prediction, RMSEP, external Q2, Kennard–Stone
condition.R            a model's observations chosen by a condition
langue.R               language of the messages (traduire("français", "English"))
modele.R, prediction.R, classification.R (HCA), nouveau.R (tables, automatic fitting)

In R: source("resources/moteur/load.R"), then for example

load.R loads the engine in English (its messages, statuses and column names). charger.R loads it in French, as before 1.0, so that scripts written for it get the same words; the French names importer_tableau(), definir_modele() and ajuster() work with either.