[bootlin/training-materials updates] master: yocto: labs: add devtool lab (2a28d878)
Luca Ceresoli
luca.ceresoli at bootlin.com
Mon Aug 7 17:39:09 CEST 2023
Repository : https://github.com/bootlin/training-materials
On branch : master
Link : https://github.com/bootlin/training-materials/commit/2a28d878d2c85b58ba167dc7244774f2eefb7681
>---------------------------------------------------------------
commit 2a28d878d2c85b58ba167dc7244774f2eefb7681
Author: Luca Ceresoli <luca.ceresoli at bootlin.com>
Date: Mon Aug 7 17:06:22 2023 +0200
yocto: labs: add devtool lab
Closes: https://github.com/bootlin/training-materials/issues/98
Signed-off-by: Luca Ceresoli <luca.ceresoli at bootlin.com>
>---------------------------------------------------------------
2a28d878d2c85b58ba167dc7244774f2eefb7681
labs/yocto-devtool/yocto-devtool.tex | 333 +++++++++++++++++++++++++
mk/yocto-stm32.mk | 4 +-
mk/yocto.mk | 4 +-
slides/yocto-devtool-lab/yocto-devtool-lab.tex | 8 +
4 files changed, 347 insertions(+), 2 deletions(-)
diff --git a/labs/yocto-devtool/yocto-devtool.tex b/labs/yocto-devtool/yocto-devtool.tex
new file mode 100644
index 00000000..8b2a2455
--- /dev/null
+++ b/labs/yocto-devtool/yocto-devtool.tex
@@ -0,0 +1,333 @@
+\subchapter{Lab9: Using devtool}{Automate recipe development and debugging
+ using devtool}
+
+During this lab, you will:
+\begin{itemize}
+ \item Use devtool to generate a new recipe more quickly
+ \item Modify a recipe to add a new patch using devtool
+ \item Upgrade a recipe to a newer version using devtool
+\end{itemize}
+
+\section{Generate a new recipe}
+
+The \code{devtool} executable is already available in your shell after
+sourcing the \code{oe-init-build-env} script. Take a look at the
+sub-commands it offers:
+\begin{bashinput}
+devtool --help
+\end{bashinput}
+
+We now want to add a new recipe for the ``GNU Hello'' program
+(\url{https://www.gnu.org/software/hello/}). We can do so using the
+\code{add} subcommand.
+
+However we want to use version 2.10 instead of the latest mainline version,
+so we can use the \code{--version} option:
+\begin{bashinput}
+devtool add --version 2.10 https://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz
+\end{bashinput}
+
+You can observe that \code{devtool} calls \code{bitbake} multiple times. In the output messages, some lines are particularly interesting:
+
+\begin{bashinput}
+INFO: Creating workspace layer in .../build/workspace
+...
+INFO: Using default source tree path .../build/workspace/sources/hello
+...
+INFO: Recipe .../build/workspace/recipes/hello/hello_2.10.bb has been automatically created; further editing may be required to make it fully functional
+\end{bashinput}
+
+The first \code{INFO} line means devtool has created the workspace in the
+\code{workspace} directory inside the \code{build} directory. Take a moment
+to inspect how the workspace looks like, but remind to never modify it
+manually: the workspace is the internal state of devtool, so it may stop
+working if it is modified externally. The workspace now looks like this:
+
+\begin{bashinput}
+$ tree workspace/ | head -n20
+workspace/
+|-- README
+|-- appends
+| `-- hello_2.10.bbappend
+|-- conf
+| `-- layer.conf
+|-- recipes
+| `-- hello
+| `-- hello_2.10.bb
+`-- sources
+ `-- hello
+ ...
+ |-- GNUmakefile
+ |-- INSTALL
+ |-- Makefile.am
+ |-- Makefile.in
+$
+\end{bashinput}
+
+As you can see the workspace is a layer, having a \code{conf/layer.conf}
+file. It also has a directory for recipes which already holds a recipe for
+GNU Hello 2.10. The \code{sources} directory contains the source code of
+the hello recipe, that devtool uses internally to manage patches.
+
+You can see that the workspace layer has been enabled by checking your
+\code{conf/bblayers.conf} or by running \code{bitbake-layers show-layers}.
+
+It's time to try building the GNU Hello program via devtool:
+
+\begin{bashinput}
+devtool build hello
+\end{bashinput}
+
+In the output messages, look for:
+\begin{bashinput}
+NOTE: hello: compiling from external source tree .../workspace/sources/hello
+\end{bashinput}
+
+This means that the recipes managed by devtool do not download the source
+code in the usual way, but rather they use the local copy of the sources
+that has been previously populated is in
+\code{workspace/sources/<RECIPENAME>}.
+
+You can have a look at the output of the build, which is in the usual
+location inside the workdir:
+\code{tmp/work/<ARCHITECTURE>/hello/2.10-r0/image/}.
+
+Using \code{devtool deploy-target} is a handy way to try the newly built
+code on the target:
+
+\begin{bashinput}
+devtool deploy-target hello root at 192.168.0.100
+\end{bashinput}
+
+This will send to the device all the file in the \code{image} subdirectory
+of the recipe work directory, keeping the directory layout and file
+permissions. You can now test the program on the target:
+
+\begin{bashinput}
+$ ssh root at 192.168.0.100
+root at bootlinlabs:~# hello
+Hello, world!
+root at bootlinlabs:~#
+\end{bashinput}
+
+Before moving the recipe to the meta-bootlinlabs layer, have a look at the
+recipe code as it has been generated by devtool:
+
+\begin{bashinput}
+devtool edit-recipe hello
+\end{bashinput}
+
+This command will open the \code{hello_2.10.bb} file of the workspace in an
+editor. You can see that the recipe has various comments added by devtool:
+you should review them, fix or adapt whatever is needed, and save the
+recipe.
+
+First of all, check the exact license by reading the first few lines of
+\code{workspace/sources/hello/src/hello.c} and you will discover it is a
+``GPL 3.0 or later''. The license guessed by devtool is
+\code{GPL-3.0-only}, thus replace it by \code{GPL-3.0-or-later}.
+
+Devtool has already computed the hashes for you, but there are several
+\code{SRC_URI} hashed, thus feel free to remove all of them except
+\code{SRC_URI[sha256sum]}.
+
+You can also simplify the \code{SRC_URI} line using \code{GNU_MIRROR},
+getting:
+\begin{verbatim}
+ SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
+\end{verbatim}
+
+Note the \code{inherit} line: from the content of the source code files of
+the GNU Hello program, devtool already guessed that it is configured using
+the Autotools and it is using GNU Gettext. This saved us a lot of time!
+There is a comment above the \code{inherit} line: do not remove it for the
+moment.
+
+Finally there is a line setting \code{EXTRA_OECONF} to an empty
+string. This line is useless unless you know you need to set some
+configuration flags, thus you can remove it together with the comment line.
+
+The resulting recipe (\code{workspace/recipes/hello/hello_2.10.bb}) should
+now look like:
+\begin{verbatim}
+ LICENSE = "GPL-3.0-or-later"
+ LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+
+ SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
+ SRC_URI[sha256sum] = "31e066137a962676e89f69d1b65382de95a7ef7d914b8cb956f41ea72e0f516b"
+
+ # NOTE: if this software is not capable of being built in a separate build directory
+ # from the source, you should replace autotools with autotools-brokensep in the
+ # inherit line
+ inherit gettext autotools
+\end{verbatim}
+
+You can double check that your recipe still works as expected using
+\code{devtool deploy-target} and when you are done you can remove the files
+from the target:
+\begin{bashinput}
+devtool undeploy-target hello root at 192.168.0.100
+\end{bashinput}
+
+You can now stop having the hello recipe handled by devtool and move it to
+the meta-bootlinlabs layer:
+\begin{bashinput}
+devtool finish -f hello ../meta-bootlinlabs/
+\end{bashinput}
+
+Now the recipe is in
+\code{../meta-bootlinlabs/recipes-hello/hello/hello_2.10.bb}. You can read
+the .bb file and verify the content has not changed. Move the recipe to a
+more reasonable directory name:
+\begin{bashinput}
+mv ../meta-bootlinlabs/recipes-hello ../meta-bootlinlabs/recipes-utils
+\end{bashinput}
+
+Now check the content of the workspace: the hello recipe is not there
+anynmore. However the source code of the GNU Hello program as still in the
+\code{workspace/sources/hello} directory. Devtool does not delete it, in
+case you have done any valuable work in it that you still haven't saved to
+a patch. As it is not your case, just delete it:
+\begin{bashinput}
+rm -fr workspace/sources/hello/
+\end{bashinput}
+
+Now double check that the recipe is still building correctly in the
+meta-bootlinlabs layer. There's no reason it should fail, right?
+\begin{bashinput}
+bitbake hello
+\end{bashinput}
+
+Oops, it failed! To have a hint about the reason, have a look at the
+comment we didn't remove from the recipe:
+\begin{verbatim}
+# NOTE: if this software is not capable of being built in a separate build directory
+# from the source, you should replace autotools with autotools-brokensep in the
+# inherit line
+\end{verbatim}
+
+This points exactly to the problem with GNU Hello 2.10: it fails building
+out-of-tree, i.e. with a build directory different from the source
+directory, as is done by default when using \code{autotools.bbclass}. Just
+fix the recipe as the comment suggests bu changing the \code{inherit}
+line. You can then remove the comment as well. Your work dir is now
+polluted so you need to clean it before running a new build:
+\begin{bashinput}
+bitbake -c clean hello
+bitbake hello
+\end{bashinput}
+
+That's all: you now have a very concise (and working!) hello recipe in the
+meta-bootlinlabs layer!
+
+\section{Modify a recipe}
+
+Now use devtool to modify the hello recipe adding a patch. This can be very
+useful if you have to fix a bug in a third-party program and there is now
+patch around to fix it yet.
+
+Use \code{devtool modify} to put an existing recipe under the control of devtool:
+\begin{bashinput}
+devtool modify hello
+\end{bashinput}
+
+Now you have two hello recipes: one in the meta-bootlinlabs layers and one
+in the workspace layer. To ensure about which will be used by bitbake, you
+can inspect the layer priorities.
+
+Now enter the source directory. You can notice devtool created a git
+repository into it:
+
+\begin{bashinput}
+cd workspace/sources/hello/
+git log
+\end{bashinput}
+
+The generated git repository contains only one commit which contains
+thepristine sources as extracted from the downloaded tarball. Now open the
+\code{src/hello.c} with an editor, go around line 60 and edit the ``Hello,
+world'' string to print whatever you prefer. Save and exit the
+editor. Check your modification using \code{git diff -- src/hello.c} and
+test it as done earlier:
+
+\begin{bashinput}
+devtool build hello
+devtool deploy-target hello root at 192.168.0.100
+\end{bashinput}
+
+Edit again the source code if needed. When you are satisfied with your
+changes, just commit them using git:
+
+\begin{bashinput}
+git add src/hello.c
+git commit -m 'Change the greeting message'
+\end{bashinput}
+
+Check your changes in the git repository, then exit the workspace:
+
+\begin{bashinput}
+git log
+cd $BUILDDIR
+\end{bashinput}
+
+You are now ready to update the original recipe to take into account your
+changes:
+
+\begin{bashinput}
+devtool update-recipe hello
+\end{bashinput}
+
+Looking at the \code{recipes-utils/hello} directory in the meta-bootlinlabs
+layer you can notice that a new patch has been created and added to
+\code{SRC_URI}. The patch applies the same change that you have just
+committed. With devtool you don't need to handle the patch syntax, but
+rather you can use git as you are probably already used to.
+
+Now remove the hello recipe from under the control of devtool, then cleanup
+as done earlier:
+
+\begin{bashinput}
+devtool reset hello
+rm -fr workspace/sources/hello/
+\end{bashinput}
+
+\section{Upgrade a recipe to the latest mainline version}
+
+devtool can be used to simplify the upgrade of a recipe to a newer mainline
+version.
+
+First, it can detect which is the latest version available on the original
+site. This is based on heuristics that work for projects that store their
+source tarballs in a standard way, which most do. To check for the latest
+version, use:
+\begin{bashinput}
+devtool latest-version hello
+\end{bashinput}
+
+At the time of this writing, the output shows that 2.12.1 is the latest
+version. You can modify the recipe to use the newest version, including the
+computation of the new hashes and renaming the .bb file, with a single
+command:
+\begin{bashinput}
+devtool upgrade hello
+\end{bashinput}
+
+Note the INFO line about license changes that you have to verify. Double
+check in the sources that the license is still ``GPL 3.0 or later'', then
+open the recipe as it is currently in the devtool workspace:
+\begin{bashinput}
+devtool edit-recipe hello
+\end{bashinput}
+
+The recipe file contains a diff between the old and the new version of the
+recipe. If you can see that the license changes are not relevant, as is the
+case for the upgrade from 2.10 to 2.12.1, then you can simply delete the
+comment, save and exit.
+
+Now check that everything works as done earlier, then apply your changes to the layer.
+\begin{bashinput}
+devtool finish hello ../meta-bootlinlabs/
+rm -fr workspace/sources/hello/
+\end{bashinput}
+
+Now the meta-bootlinlabs layer contains the latest version of GNU Hello!
diff --git a/mk/yocto-stm32.mk b/mk/yocto-stm32.mk
index 407e567f..38e8ca30 100644
--- a/mk/yocto-stm32.mk
+++ b/mk/yocto-stm32.mk
@@ -28,6 +28,7 @@ YOCTO_STM32_SLIDES = \
yocto-sdk \
yocto-sdk-lab \
yocto-devtool \
+ yocto-devtool-lab \
yocto-layer-management \
yocto-runtime-package-management \
yocto-resources \
@@ -42,4 +43,5 @@ YOCTO_STM32_LABS = setup \
yocto-extend-recipe \
yocto-custom-machine \
yocto-custom-image \
- yocto-sdk
+ yocto-sdk \
+ yocto-devtool
diff --git a/mk/yocto.mk b/mk/yocto.mk
index fa19767b..3e6990e7 100644
--- a/mk/yocto.mk
+++ b/mk/yocto.mk
@@ -28,6 +28,7 @@ YOCTO_SLIDES = \
yocto-sdk \
yocto-sdk-lab \
yocto-devtool \
+ yocto-devtool-lab \
yocto-layer-management \
yocto-runtime-package-management \
yocto-resources \
@@ -42,4 +43,5 @@ YOCTO_LABS = setup \
yocto-extend-recipe \
yocto-custom-machine \
yocto-custom-image \
- yocto-sdk
+ yocto-sdk \
+ yocto-devtool
diff --git a/slides/yocto-devtool-lab/yocto-devtool-lab.tex b/slides/yocto-devtool-lab/yocto-devtool-lab.tex
new file mode 100644
index 00000000..e406f56e
--- /dev/null
+++ b/slides/yocto-devtool-lab/yocto-devtool-lab.tex
@@ -0,0 +1,8 @@
+\setuplabframe{Using devtool}
+{
+ \begin{itemize}
+ \item Generate a new recipe
+ \item Modify a recipe to add a new patch
+ \item Upgrade a recipe to a newer version
+ \end{itemize}
+}
More information about the training-materials-updates
mailing list