OctoPi build now supports docker

guysoft · November 25, 2018, 4:41pm

Hey all,
I recently added docker support to building OctoPi's image. This is manly useful if you want to run OctoPi builds in the cloud, since vagrant is usually not supported by cloud providers.

instructions available here.

If there are any people here building their custom OctoPi build I would value feedback if this helps you.
OctoPi's nightly started using this yesterday.

OutsourcedGuru · November 25, 2018, 7:27pm

Brilliant. I must say that I'm impressed at how fast Docker is becoming the go-to virtualization of choice by I.T. people over VM technology, say.

That said, it might be worth backing up and helping people to understand what Docker is and what it's good for. Pretty sure I've done a little work with this already just to see if I could get OctoPrint running in a container and managed to do so. There are still times when I scratch my head and wonder exactly in what cases I'd want to do this.

guysoft · November 26, 2018, 2:24pm

Docker is not virtualization, its container management, some call it "containerization".
OctoPi build also works with vagrant which is virtualization.

This mostly enables easy setup of OctoPi/CustomPiOS builds on cloud providers that do not allow running virtualbox. Also it would mean you could automated an OctoPi, or raspberry OS builds in general in the cloud.

tedder42 · November 26, 2018, 3:49pm

I spin up OctoPrint in a container to do bugfixes and the like. virtualenv makes it a little messy but it's okay.

OutsourcedGuru · November 26, 2018, 8:03pm

See, this is where I'm trying to help to define what Docker is/does. In my own head, I think of the Docker container as being the equivalent of the virtualenv compartmentalization so much so that you'd think that you could remove that step completely.

This example seems to back me up on that assertion. Note that there's no mention of virtualenv, installing it or invoking it.

In trying to learn Docker, I imagine trying to teach others what it is and how it works and I must admit that I'm still trying to wrap my head around the concepts:

I would suggest that a Docker OctoPi image creation results in something that would be like cloning a working OctoPrint-on-Pi platform, removing things that are unnecessary and then converting it into something that works like a stand-alone app. It's just created in the reverse order with a minimalistic operating system stub, if you will.

To this, I'd add that the image runs within the expected (included) operating system which is quite often different from the operating system of the computer which might invoke it, Docker then handles this.

guysoft · November 27, 2018, 5:50pm

This is a container to build OctoPi, not an OctoPrint container. Please do not mix between them.
There is already an OctoPrint container.

OutsourcedGuru · November 27, 2018, 5:59pm

Do you know if anyone has a USB-based device of some kind which 1) presents GPIO pins to a computer like OSX/Linux/Windows and 2) which contains drivers so that one could spin up a Docker/OctoPi container on a Mac, let's say, and run something like the Enclosure plugin?

I like the idea of developing here on my MacBook but I really need the Raspbian toolstack and native GPIO hardware/behavior.

guysoft · December 8, 2018, 12:37pm

This is as easy now as:

Install Docker
Install docker-compose
git clone https://github.com/guysoft/OctoPi.git
cd OctoPi/src
wget https://raw.githubusercontent.com/guysoft/CustomPiOS/devel/src/docker/docker-compose.yml
wget --directory-prefix=./image -c --trust-server-names 'https://downloads.raspberrypi.org/raspbian_lite_latest'
echo "DISTRO_NAME=octopi" > .env
sudo docker-compose up -d
sudo docker exec -it octopi-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

OutsourcedGuru · December 8, 2018, 9:53pm

(No time like the present.) This is my version running in OSX. Since I didn't have wget and was being hard-headed, I converted things for curl below. I store things like this under a ~/sites subdirectory under my home folder.

I already had Docker and the Docker Toolbox installed (which includes Compose)
I did, though, check for updates and applied them (which included an update to Compose) with a relaunch afterwards
cd ~/sites && git clone https://github.com/guysoft/OctoPi.git && cd ~/sites/OctoPi/src # Pull Guy's OctoPi image
curl -o docker-compose.yaml https://raw.githubusercontent.com/guysoft/CustomPiOS/devel/src/docker/docker-compose.yml # Pull his yaml file for Docker to configure the container and to point to the image

version: '3.6'

services:
  custompios:
    image: guysoft/custompios:devel
    container_name: ${DISTRO_NAME}-build
    tty: true
    restart: always
    privileged: true
    volumes:
      - ./:/distro
    devices:
      - /dev/loop-control
    labels:
      - com.centurylinklabs.watchtower.enable=true

curl -v https://downloads.raspberrypi.org/raspbian_lite_latest # To determine the latest version, using it in the command below
curl -o image/2018-11-13-raspbian-stretch-lite.zip http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-11-15/2018-11-13-raspbian-stretch-lite.zip
echo "DISTRO_NAME=octopi-build" > .env
sudo docker-compose up -d # More -> https://docs.docker.com/compose/reference/up/

Password: **********
Creating network "src_default" with the default driver
Pulling custompios (guysoft/custompios:devel)...
devel: Pulling from guysoft/custompios
54f7e8ac135a: Pull complete
29ac05956748: Pull complete [********time for some tea*********]
61f1e9636e0a: Pull complete
Creating octopi-build-build ... done

sudo docker exec -it octopi-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

Error: No such container: octopi-build

sudo docker exec -it octopi-build-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

+ set -e
+ whoami
root
+ export DIST_PATH=/distro
+ DIST_PATH=/distro
+ export CUSTOM_PI_OS_PATH=/CustomPiOS
+ CUSTOM_PI_OS_PATH=/CustomPiOS
+ '[' 0 -gt 0 ']'
+ BUILD_VARIANT=default
+ WORKSPACE_POSTFIX=
+ '[' default '!=' default ']'
++ lsof /distro/workspace/mount
++ awk '{print $2}'
lsof: status error on /distro/workspace/mount: No such file or directory
lsof 4.89
 latest revision: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/
 latest FAQ: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/FAQ
 latest man page: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/lsof_man
 usage: [-?abhKlnNoOPRtUvVX] [+|-c c] [+|-d s] [+D D] [+|-E] [+|-e s] [+|-f[gG]]
 [-F [f]] [-g [s]] [-i [i]] [+|-L [l]] [+m [m]] [+|-M] [-o [o]] [-p s]
 [+|-r [t]] [-s [p:s]] [-S [t]] [-T [t]] [-u s] [+|-w] [-x [fl]] [--] [names]
Use the ``-h'' option to get more help information.
+ rm '/distro/workspace/*.img'
rm: cannot remove '/distro/workspace/*.img': No such file or directory
+ true
+ rm '/distro/workspace/*.raw'
+ true
rm: cannot remove '/distro/workspace/*.raw': No such file or directory
+ rm '/distro/workspace/*.zip'
rm: cannot remove '/distro/workspace/*.zip': No such file or directory
+ true
+ pushd /distro
/distro /
+ umount /distro/workspace/mount/boot
umount: /distro/workspace/mount/boot: mountpoint not found
+ true
+ umount /distro/workspace/mount/dev/pts
umount: /distro/workspace/mount/dev/pts: mountpoint not found
+ true
+ umount /distro/workspace/mount
umount: /distro/workspace/mount: mountpoint not found
+ true
+ /CustomPiOS/build_custom_os
+ echo 'Distro path: /distro'
Distro path: /distro
+ echo 'CustomPiOS path: /CustomPiOS'
CustomPiOS path: /CustomPiOS
+ echo ================================================================
================================================================
+++ dirname /CustomPiOS/build_custom_os
++ cd /CustomPiOS
++ pwd
+ DIR=/CustomPiOS
+ /CustomPiOS/build
+ '[' -n '' ']'
+ LOG=build.log
+ define SCRIPT
+ IFS='\n'
+ read -r -d '' SCRIPT
+ true
+ '[' build.log '!=' no ']'
+ touch build.log
+ chmod 666 build.log
+ eval 'BUILD_SCRIPT_PATH=$(dirname $(realpath -s $BASH_SOURCE))
source ${BUILD_SCRIPT_PATH}/common.sh
install_cleanup_trap

CUSTOM_OS_PATH=$(dirname $(realpath -s $0))
pushd $CUSTOM_OS_PATH
  export OCTOPI_COMMIT=`git rev-parse HEAD`
popd

source ${CUSTOM_PI_OS_PATH}/config ${@}
${CUSTOM_PI_OS_PATH}/config_sanity

[ "$CONFIG_ONLY" == "yes" ] || source ${CUSTOM_OS_PATH}/custompios
'
+ tee build.log
++++ realpath -s /CustomPiOS/build
+++ dirname /CustomPiOS/build
++ BUILD_SCRIPT_PATH=/CustomPiOS
++ source /CustomPiOS/common.sh
++ install_cleanup_trap
++ set -e
++ trap cleanup SIGINT SIGTERM
++++ realpath -s /CustomPiOS/build
+++ dirname /CustomPiOS/build
++ CUSTOM_OS_PATH=/CustomPiOS
++ pushd /CustomPiOS
/CustomPiOS /distro
+++ git rev-parse HEAD
fatal: Not a git repository (or any of the parent directories): .git
++ export OCTOPI_COMMIT=
++ OCTOPI_COMMIT=
++ popd
/distro
++ source /CustomPiOS/config
+++++ realpath -s /CustomPiOS/config
++++ dirname /CustomPiOS/config
+++ CONFIG_DIR=/CustomPiOS
+++ BUILD_VARIANT=default
+++ BUILD_FLAVOR=default
+++ WORKSPACE_POSTFIX=
+++ '[' 0 -gt 0 ']'
+++ '[' 0 -gt 1 ']'
+++ '[' default '!=' default ']'
+++ echo -e '--> Building VARIANT default, FLAVOR default'
--> Building VARIANT default, FLAVOR default
+++ '[' -n '' ']'
+++ '[' -n '' ']'
+++ '[' -f /CustomPiOS/config.local ']'
+++ source /distro/config
++++ export DIST_NAME=OctoPi
++++ DIST_NAME=OctoPi
++++ export DIST_VERSION=0.16.0
++++ DIST_VERSION=0.16.0
++++ export 'MODULES=base(raspicam, network, disable-services(octopi), password-for-sudo)'
++++ MODULES='base(raspicam, network, disable-services(octopi), password-for-sudo)'
+++ '[' -f /distro/config.local ']'
+++ TMP='base,raspicam, network, disable-services,octopi), password-for-sudo)'
+++ TMP='base,raspicam,network,disable-services,octopi),password-for-sudo)'
+++ MODULES_LIST=base,raspicam,network,disable-services,octopi,,password-for-sudo,
++++ echo base,raspicam,network,disable-services,octopi,,password-for-sudo,
++++ tr , '\n'
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/base ']'
+++ '[' -d /CustomPiOS/modules/base ']'
+++ export MODULE_PATH=/CustomPiOS/modules/base
+++ MODULE_PATH=/CustomPiOS/modules/base
+++ echo loading base config at /CustomPiOS/modules/base/config
loading base config at /CustomPiOS/modules/base/config
+++ '[' -f /CustomPiOS/modules/base/config ']'
+++ source /CustomPiOS/modules/base/config
++++ BASE_VERSION=1.1.0
++++ '[' -n '' ']'
++++ BASE_PRESCRIPT=
++++ '[' -n '' ']'
++++ BASE_POSTSCRIPT=
++++ '[' -n '' ']'
++++ BASE_IMAGE_PATH=/distro/image
++++ '[' -n '' ']'
++++ BASE_IMAGE_RASPBIAN=yes
++++ '[' -n '' ']'
+++++ head -n 1
+++++ ls -t /distro/image/2018-11-13-raspbian-stretch-lite.zip
++++ BASE_ZIP_IMG=/distro/image/2018-11-13-raspbian-stretch-lite.zip
++++ '[' -n '' ']'
++++ BASE_WORKSPACE=/distro/workspace
++++ '[' -n '' ']'
++++ BASE_MOUNT_PATH=/distro/workspace/mount
++++ '[' -n '' ']'
++++ BASE_ROOT_PARTITION=2
++++ '[' -n '' ']'
++++ BASE_IMAGE_ENLARGEROOT=400
++++ '[' -n '' ']'
++++ BASE_IMAGE_RESIZEROOT=200
++++ '[' -n '' ']'
++++ BASE_APT_CACHE=/distro/workspace/aptcache
++++ '[' -n '' ']'
++++ BASE_APT_PROXY=
++++ '[' -n '' ']'
++++ BASE_PYPI_INDEX=
++++ '[' -n '' ']'
++++ BASE_OVERRIDE_HOSTNAME=octopi
++++ '[' -n '' ']'
++++ BASE_BUILD_REPO_MIRROR=
++++ '[' -n '' ']'
++++ BASE_SSH_ENABLE=yes
++++ '[' -n '' ']'
+++++ pushd /CustomPiOS
+++++ git rev-parse HEAD
fatal: Not a git repository (or any of the parent directories): .git
+++++ popd
++++ BASE_COMMIT=
++++ '[' -n '' ']'
++++ BASE_CONFIG_MEMSPLIT=default
++++ '[' -n '' ']'
++++ BASE_CONFIG_TIMEZONE=default
++++ '[' -n '' ']'
++++ BASE_CONFIG_LOCALE=default
++++ '[' -n '' ']'
++++ BASE_CONFIG_KEYBOARD=default
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=BASE
+++ module_up=BASE_
+++ IFS=
+++ read -r var
++++ compgen -A variable BASE_
+++ export BASE_APT_CACHE
+++ echo BASE_APT_CACHE
BASE_APT_CACHE
+++ IFS=
+++ read -r var
+++ export BASE_APT_PROXY
+++ echo BASE_APT_PROXY
BASE_APT_PROXY
+++ IFS=
+++ read -r var
+++ export BASE_BUILD_REPO_MIRROR
+++ echo BASE_BUILD_REPO_MIRROR
BASE_BUILD_REPO_MIRROR
+++ IFS=
+++ read -r var
+++ export BASE_COMMIT
+++ echo BASE_COMMIT
BASE_COMMIT
+++ IFS=
+++ read -r var
+++ export BASE_CONFIG_KEYBOARD
+++ echo BASE_CONFIG_KEYBOARD
BASE_CONFIG_KEYBOARD
+++ IFS=
+++ read -r var
+++ export BASE_CONFIG_LOCALE
+++ echo BASE_CONFIG_LOCALE
BASE_CONFIG_LOCALE
+++ IFS=
+++ read -r var
+++ export BASE_CONFIG_MEMSPLIT
+++ echo BASE_CONFIG_MEMSPLIT
BASE_CONFIG_MEMSPLIT
+++ IFS=
+++ read -r var
+++ export BASE_CONFIG_TIMEZONE
+++ echo BASE_CONFIG_TIMEZONE
BASE_CONFIG_TIMEZONE
+++ IFS=
+++ read -r var
+++ export BASE_IMAGE_ENLARGEROOT
+++ echo BASE_IMAGE_ENLARGEROOT
BASE_IMAGE_ENLARGEROOT
+++ IFS=
+++ read -r var
+++ export BASE_IMAGE_PATH
+++ echo BASE_IMAGE_PATH
BASE_IMAGE_PATH
+++ IFS=
+++ read -r var
+++ export BASE_IMAGE_RASPBIAN
+++ echo BASE_IMAGE_RASPBIAN
BASE_IMAGE_RASPBIAN
+++ IFS=
+++ read -r var
+++ export BASE_IMAGE_RESIZEROOT
+++ echo BASE_IMAGE_RESIZEROOT
BASE_IMAGE_RESIZEROOT
+++ IFS=
+++ read -r var
+++ export BASE_MOUNT_PATH
+++ echo BASE_MOUNT_PATH
BASE_MOUNT_PATH
+++ IFS=
+++ read -r var
+++ export BASE_OVERRIDE_HOSTNAME
+++ echo BASE_OVERRIDE_HOSTNAME
BASE_OVERRIDE_HOSTNAME
+++ IFS=
+++ read -r var
+++ export BASE_POSTSCRIPT
+++ echo BASE_POSTSCRIPT
BASE_POSTSCRIPT
+++ IFS=
+++ read -r var
+++ export BASE_PRESCRIPT
+++ echo BASE_PRESCRIPT
BASE_PRESCRIPT
+++ IFS=
+++ read -r var
+++ export BASE_PYPI_INDEX
+++ echo BASE_PYPI_INDEX
BASE_PYPI_INDEX
+++ IFS=
+++ read -r var
+++ export BASE_ROOT_PARTITION
+++ echo BASE_ROOT_PARTITION
BASE_ROOT_PARTITION
+++ IFS=
+++ read -r var
+++ export BASE_SSH_ENABLE
+++ echo BASE_SSH_ENABLE
BASE_SSH_ENABLE
+++ IFS=
+++ read -r var
+++ export BASE_VERSION
+++ echo BASE_VERSION
BASE_VERSION
+++ IFS=
+++ read -r var
+++ export BASE_WORKSPACE
+++ echo BASE_WORKSPACE
BASE_WORKSPACE
+++ IFS=
+++ read -r var
+++ export BASE_ZIP_IMG
+++ echo BASE_ZIP_IMG
BASE_ZIP_IMG
+++ IFS=
+++ read -r var
+++ echo ================================================================
================================================================
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/raspicam ']'
+++ '[' -d /CustomPiOS/modules/raspicam ']'
+++ export MODULE_PATH=/CustomPiOS/modules/raspicam
+++ MODULE_PATH=/CustomPiOS/modules/raspicam
+++ echo loading raspicam config at /CustomPiOS/modules/raspicam/config
loading raspicam config at /CustomPiOS/modules/raspicam/config
+++ '[' -f /CustomPiOS/modules/raspicam/config ']'
+++ source /CustomPiOS/modules/raspicam/config
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=RASPICAM
+++ module_up=RASPICAM_
+++ IFS=
+++ read -r var
++++ compgen -A variable RASPICAM_
+++ echo ================================================================
================================================================
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/network ']'
+++ '[' -d /CustomPiOS/modules/network ']'
+++ export MODULE_PATH=/CustomPiOS/modules/network
+++ MODULE_PATH=/CustomPiOS/modules/network
+++ echo loading network config at /CustomPiOS/modules/network/config
loading network config at /CustomPiOS/modules/network/config
+++ '[' -f /CustomPiOS/modules/network/config ']'
+++ source /CustomPiOS/modules/network/config
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=NETWORK
+++ module_up=NETWORK_
+++ IFS=
+++ read -r var
++++ compgen -A variable NETWORK_
+++ echo ================================================================
================================================================
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/disable-services ']'
+++ '[' -d /CustomPiOS/modules/disable-services ']'
+++ export MODULE_PATH=/CustomPiOS/modules/disable-services
+++ MODULE_PATH=/CustomPiOS/modules/disable-services
+++ echo loading disable-services config at /CustomPiOS/modules/disable-services/config
loading disable-services config at /CustomPiOS/modules/disable-services/config
+++ '[' -f /CustomPiOS/modules/disable-services/config ']'
+++ echo 'WARNING: module disable-services has no config file'
WARNING: module disable-services has no config file
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=DISABLE-SERVICES
+++ module_up=DISABLE_SERVICES_
+++ IFS=
+++ read -r var
++++ compgen -A variable DISABLE_SERVICES_
+++ echo ================================================================
================================================================
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/octopi ']'
+++ export MODULE_PATH=/distro/modules/octopi
+++ MODULE_PATH=/distro/modules/octopi
+++ echo loading octopi config at /distro/modules/octopi/config
loading octopi config at /distro/modules/octopi/config
+++ '[' -f /distro/modules/octopi/config ']'
+++ source /distro/modules/octopi/config
++++ '[' -n '' ']'
+++++ grep zipball_url
+++++ cut -d : -f 2,3
+++++ tr -d ,
+++++ tr -d '"'
+++++ wget -q -O - https://api.github.com/repos/foosel/OctoPrint/releases/latest
++++ OCTOPI_OCTOPRINT_ARCHIVE=' https://api.github.com/repos/foosel/OctoPrint/zipball/1.3.9'
++++ '[' -n '' ']'
++++ OCTOPI_OCTOPRINT_REPO_SHIP=https://github.com/foosel/OctoPrint.git
++++ '[' -n '' ']'
++++ OCTOPI_INCLUDE_OCTOPRINT=yes
++++ '[' -n '' ']'
++++ OCTOPI_PYBONJOUR_ARCHIVE=https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/pybonjour/pybonjour-1.1.1.tar.gz
++++ '[' -n '' ']'
++++ OCTOPI_CURAENGINE_VERSION=15.04.6
++++ '[' -n '' ']'
++++ OCTOPI_CURAENGINE_ARCHIVE=https://github.com/Ultimaker/CuraEngine/archive/15.04.6.zip
++++ '[' -n '' ']'
++++ OCTOPI_INCLUDE_CURAENGINE=yes
++++ '[' -n '' ']'
++++ OCTOPI_MJPGSTREAMER_REPO_SHIP=https://github.com/jacksonliam/mjpg-streamer.git
++++ '[' -n '' ']'
++++ OCTOPI_MJPGSTREAMER_REPO_BUILD=
++++ '[' -n '' ']'
++++ OCTOPI_MJPGSTREAMER_REPO_BRANCH=master
++++ '[' -n '' ']'
++++ OCTOPI_MJPGSTREAMER_REPO_DEPTH=1
++++ '[' -n '' ']'
++++ OCTOPI_INCLUDE_MJPGSTREAMER=yes
++++ '[' -n '' ']'
++++ OCTOPI_INCLUDE_HAPROXY=yes
++++ '[' -n '' ']'
++++ OCTOPI_INCLUDE_WIRINGPI=yes
++++ '[' -n '' ']'
+++++ tr -d '"'
+++++ grep browser_download_url
+++++ tr -d ,
+++++ wget -q -O - https://api.github.com/repos/mikefarah/yq/releases/latest
+++++ cut -d : -f 2,3
+++++ grep yq_linux_arm
++++ OCTOPI_YQ_DOWNLOAD=' https://github.com/mikefarah/yq/releases/download/2.2.0/yq_linux_arm
 https://github.com/mikefarah/yq/releases/download/2.2.0/yq_linux_arm64'
++++ '[' -n '' ']'
+++++ pushd /distro
+++++ git rev-parse HEAD
fatal: Not a git repository (or any parent up to mount point /distro)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+++++ popd
++++ OCTOPI_COMMIT=
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=OCTOPI
+++ module_up=OCTOPI_
+++ IFS=
+++ read -r var
++++ compgen -A variable OCTOPI_
+++ export OCTOPI_COMMIT
+++ echo OCTOPI_COMMIT
OCTOPI_COMMIT
+++ IFS=
+++ read -r var
+++ export OCTOPI_CURAENGINE_ARCHIVE
+++ echo OCTOPI_CURAENGINE_ARCHIVE
OCTOPI_CURAENGINE_ARCHIVE
+++ IFS=
+++ read -r var
+++ export OCTOPI_CURAENGINE_VERSION
+++ echo OCTOPI_CURAENGINE_VERSION
OCTOPI_CURAENGINE_VERSION
+++ IFS=
+++ read -r var
+++ export OCTOPI_INCLUDE_CURAENGINE
+++ echo OCTOPI_INCLUDE_CURAENGINE
OCTOPI_INCLUDE_CURAENGINE
+++ IFS=
+++ read -r var
+++ export OCTOPI_INCLUDE_HAPROXY
+++ echo OCTOPI_INCLUDE_HAPROXY
OCTOPI_INCLUDE_HAPROXY
+++ IFS=
+++ read -r var
+++ export OCTOPI_INCLUDE_MJPGSTREAMER
+++ echo OCTOPI_INCLUDE_MJPGSTREAMER
OCTOPI_INCLUDE_MJPGSTREAMER
+++ IFS=
+++ read -r var
+++ export OCTOPI_INCLUDE_OCTOPRINT
+++ echo OCTOPI_INCLUDE_OCTOPRINT
OCTOPI_INCLUDE_OCTOPRINT
+++ IFS=
+++ read -r var
+++ export OCTOPI_INCLUDE_WIRINGPI
+++ echo OCTOPI_INCLUDE_WIRINGPI
OCTOPI_INCLUDE_WIRINGPI
+++ IFS=
+++ read -r var
+++ export OCTOPI_MJPGSTREAMER_REPO_BRANCH
+++ echo OCTOPI_MJPGSTREAMER_REPO_BRANCH
OCTOPI_MJPGSTREAMER_REPO_BRANCH
+++ IFS=
+++ read -r var
+++ export OCTOPI_MJPGSTREAMER_REPO_BUILD
+++ echo OCTOPI_MJPGSTREAMER_REPO_BUILD
OCTOPI_MJPGSTREAMER_REPO_BUILD
+++ IFS=
+++ read -r var
+++ export OCTOPI_MJPGSTREAMER_REPO_DEPTH
+++ echo OCTOPI_MJPGSTREAMER_REPO_DEPTH
OCTOPI_MJPGSTREAMER_REPO_DEPTH
+++ IFS=
+++ read -r var
+++ export OCTOPI_MJPGSTREAMER_REPO_SHIP
+++ echo OCTOPI_MJPGSTREAMER_REPO_SHIP
OCTOPI_MJPGSTREAMER_REPO_SHIP
+++ IFS=
+++ read -r var
+++ export OCTOPI_OCTOPRINT_ARCHIVE
+++ echo OCTOPI_OCTOPRINT_ARCHIVE
OCTOPI_OCTOPRINT_ARCHIVE
+++ IFS=
+++ read -r var
+++ export OCTOPI_OCTOPRINT_REPO_SHIP
+++ echo OCTOPI_OCTOPRINT_REPO_SHIP
OCTOPI_OCTOPRINT_REPO_SHIP
+++ IFS=
+++ read -r var
+++ export OCTOPI_PYBONJOUR_ARCHIVE
+++ echo OCTOPI_PYBONJOUR_ARCHIVE
OCTOPI_PYBONJOUR_ARCHIVE
+++ IFS=
+++ read -r var
+++ export OCTOPI_YQ_DOWNLOAD
+++ echo OCTOPI_YQ_DOWNLOAD
OCTOPI_YQ_DOWNLOAD
+++ IFS=
+++ read -r var
+++ echo ================================================================
================================================================
+++ for module in $(echo "${MODULES_LIST}" | tr "," "\n")
+++ '[' -d /distro/modules/password-for-sudo ']'
+++ '[' -d /CustomPiOS/modules/password-for-sudo ']'
+++ export MODULE_PATH=/CustomPiOS/modules/password-for-sudo
+++ MODULE_PATH=/CustomPiOS/modules/password-for-sudo
+++ echo loading password-for-sudo config at /CustomPiOS/modules/password-for-sudo/config
loading password-for-sudo config at /CustomPiOS/modules/password-for-sudo/config
+++ '[' -f /CustomPiOS/modules/password-for-sudo/config ']'
+++ echo 'WARNING: module password-for-sudo has no config file'
WARNING: module password-for-sudo has no config file
+++ echo ================================================================
================================================================
+++ echo 'Using the following config:'
Using the following config:
+++ module_up=PASSWORD-FOR-SUDO
+++ module_up=PASSWORD_FOR_SUDO_
+++ IFS=
+++ read -r var
++++ compgen -A variable PASSWORD_FOR_SUDO_
+++ echo ================================================================
================================================================
++ /CustomPiOS/config_sanity
++ '[' '' == yes ']'
++ source /CustomPiOS/custompios
+++ set -e
+++ export LC_ALL=C
+++ LC_ALL=C
+++ source /CustomPiOS/common.sh
+++ mkdir -p /distro/workspace
+++ mkdir -p /distro/workspace/mount
+++ rm -rf '/distro/workspace/*.tar.gz'
+++ install_cleanup_trap
+++ set -e
+++ trap cleanup SIGINT SIGTERM
+++ install_fail_on_error_trap
+++ set -e
+++ trap 'echo_red "build failed, unmounting image..." && cd $DIST_PATH && ( unmount_image $BASE_MOUNT_PATH force || true ) && echo_red -e "\nBUILD FAILED!\n"' ERR
+++ unmount_image /distro/workspace/mount force
+++ mount_path=/distro/workspace/mount
+++ force=
+++ '[' 2 -gt 1 ']'
+++ force=force
+++ '[' -n force ']'
++++ sudo lsof /distro/workspace/mount
++++ awk '{print $2}'
++++ sudo mount
++++ awk -F ' on ' '{print $2}'
++++ awk '{print $1}'
++++ sort -r
++++ grep /distro/workspace/mount
+++ pushd /distro/workspace
/distro/workspace /distro
+++ '[' -e '*.img' ']'
+++ '[' '!' -f /distro/image/2018-11-13-raspbian-stretch-lite.zip ']'
+++ '[' /distro/image/2018-11-13-raspbian-stretch-lite.zip == '' ']'
+++ 7za x -aoa /distro/image/2018-11-13-raspbian-stretch-lite.zip

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C,Utf16=off,HugeFiles=on,64 bits,2 CPUs Intel(R) Core(TM) i5-5350U CPU @ 1.80GHz (306D4),ASM,AES-NI)

Scanning the drive for archives:
1 file, 399 bytes (1 KiB)

Extracting archive: /distro/image/2018-11-13-raspbian-stretch-lite.zip
ERROR: /distro/image/2018-11-13-raspbian-stretch-lite.zip
/distro/image/2018-11-13-raspbian-stretch-lite.zip
Open ERROR: Can not open the file as [zip] archive


ERRORS:
Is not archive

Can't open as archive: 1
Files: 0
Size:       0
Compressed: 0
++++ echo_red 'build failed, unmounting image...'
++++ echo -e -n '\e[91m'
++++ echo build failed, unmounting image...
build failed, unmounting image...
++++ echo -e -n '\e[0m'
++++ cd /distro
++++ unmount_image /distro/workspace/mount force
++++ mount_path=/distro/workspace/mount
++++ force=
++++ '[' 2 -gt 1 ']'
++++ force=force
++++ '[' -n force ']'
+++++ sudo lsof /distro/workspace/mount
+++++ awk '{print $2}'
+++++ sudo mount
+++++ grep /distro/workspace/mount
+++++ awk -F ' on ' '{print $2}'
+++++ awk '{print $1}'
+++++ sort -r
++++ echo_red -e '\nBUILD FAILED!\n'
++++ echo -e -n '\e[91m'
++++ echo -e '\nBUILD' 'FAILED!\n'

BUILD FAILED!

++++ echo -e -n '\e[0m'
+ exit 1

So I got a failed build on that. Note that my build appended another "-build" at the end of that. Any thoughts?

build.log (16.6 KB)

I would guess:

the yaml needs to lose that "-build"
something needs to change a path from "/distro/workspace/mount" to "./workspace/mount"

guysoft · December 9, 2018, 1:50am

Ok.
I fixed step 6 to remove the extra -build

It seems you downloaded 2018-11-13-raspbian-stretch-lite.zip in a diffrent way than the one I gave, and its not a valid zip file.

The console says quite clearly:

Extracting archive: /distro/image/2018-11-13-raspbian-stretch-lite.zip
ERROR: /distro/image/2018-11-13-raspbian-stretch-lite.zip
/distro/image/2018-11-13-raspbian-stretch-lite.zip
Open ERROR: Can not open the file as [zip] archive

Is not archive

Can't open as archive: 1

Take a look at it and make sure you have a valid raspbian zip archive there.

OutsourcedGuru · December 9, 2018, 4:57am

I'm noting that my step #3 above is in error. Notice that I've accidentally renamed docker-compose.yml to docker-compose.yaml during the download. I'm surprised that it got as far as it did.

I also note that the docker-compose.yml file has some file attribute decoration now that it didn't have before:

-rw-r--r--  1 me  staff    306 Dec  8 13:17 docker-compose.yaml
-rw-r--r--@ 1 me  staff    306 Dec  8 20:11 docker-compose.yml

Presumably this is an ACL that's being identified.

$ ls -l -@ docker-compose.yml
-rw-r--r--@ 1 me  staff  306 Dec  8 20:11 docker-compose.yml
	user.xdg.origin.url	 88

Manually edited my .env file, removing the superfluous "-build" ending
Did a brew install wget
Removed the src/image/*.zip file
Overwrote the yaml with wget https://raw.githubusercontent.com/guysoft/CustomPiOS/devel/src/docker/docker-compose.yml, deleting the previous attempt
Downloaded the image again with wget --directory-prefix=./image -c --trust-server-names 'https://downloads.raspberrypi.org/raspbian_lite_latest'
Re-ran sudo docker-compose up -d
Re-ran sudo docker exec -it octopi-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

It seems to be a lot happier this time.

I note that I see a few of these throughout the build log: sudo: unable to resolve host c12ac21c06dd. Presumably this would be some dynamically-provided hostname for the spun up host. More info

Before each of those, I'm seeing paragraphs like (hard to say if they harm anything):

Unknown host QEMU_IFLA type: 40
Unknown host QEMU_IFLA type: 41
Unknown host QEMU_IFLA type: 40
Unknown host QEMU_IFLA type: 41
Unknown QEMU_IFLA_INFO_KIND ipip
Unknown host QEMU_IFLA type: 40
Unknown host QEMU_IFLA type: 41
Unknown QEMU_IFLA_INFO_KIND ip6tnl
Unknown host QEMU_IFLA type: 40
Unknown host QEMU_IFLA type: 41

Seemed to be an error making the ssl-cert (DNS-related for similar reasons as above).

Looks like I have a new zip file in the workspace folder!

build.log (210.7 KB)

guysoft · December 9, 2018, 3:29pm

Congrats! You got a build of OctoPi! You can build for other devies apart from RaspberryPi too like this. And you can build not any distro using CustomPiOS in general.

I am not sure how to fix those errors on Unknown host QEMU_IFLA type: 40.
They started showing up after a newer version of qemu, if you figure out how to fix them you are welcome to PR. But the build seems to work fine.

OutsourcedGuru · December 9, 2018, 4:58pm

The only error that seemed to cause actual harm would be the SSL certification creation. There was a suggestion in the log to try again using different parameters. This would only effect someone who later would like to run https.

guysoft · December 10, 2018, 9:29am

SSL generation is ok, because its generated at boot.
There is a script on raspbian /lib/systemd/system/regenerate_ssh_host_keys.service.

OutsourcedGuru · December 10, 2018, 5:41pm

I'll research the other things to see if they're pertinent or if there's some means of quieting the log.

This is great, btw.

guysoft · December 10, 2018, 6:04pm

There is none at the moment, open bug at:

https://github.com/guysoft/CustomPiOS/issues/1

There is also direcrion how ot might be written.

reloxx13 · January 13, 2019, 12:05am

this is everyting but not easy and thats why docker is not accepted by new/noob users.

its bad practicise of delivering a docker container.

guysoft · January 13, 2019, 4:31pm

@reloxx13 So don't use it, there are two other methods totalling at 3 different ways you can build CustomPiOS now.

tedder42 · February 8, 2019, 10:34pm

I think I posted on a ticket to you, but I have a couple of "home servers" that are pretty much just sitting there, idling away, running Docker on linux. If we ran the builds in parallel on my machines we'd have a backup to reduce the "build outage" events.

guysoft · February 10, 2019, 2:04pm

Sure, ping me details and I will set it up.
I didn't see any issue, you are welcome to link to it.