Deployment on GitLab

Development
1

Hello,

I’m trying to set up a CI integration via GitLab and I’m having compilation issues!
I based it on the Circle CI pipeline
If you have any ideas, I’m all ears! :slight_smile:
It seems to be coming from bcrypt, I tried to restart the compilation but it still fails.

I have a problem with the test_server and build_server tasks

The tasks in question:

test_server:
  image: node:12
  stage: test
  script:
      # Install system dependencies 
    - apt-get update && apt-get install -y libudev-dev sqlite3 openssl openzwave libopenzwave1.5-dev build-essential python
      # Install global dependencies
    - npm install typescript node-gyp npm@latest -g
    - cd server
      # Install NPM dependencies
    - npm install
    - npm rebuild bcrypt --build-from-source
      # Run prettier
    - npm run prettier-check
      # Run eslint
    - npm run eslint
      # Run tests with coverage
    - npm run coverage

.build_job_template: &build_job_definition
  image: docker:18.06.3-ce-git
  stage: deploy
  script:
      # Login to Gitlab Docker Registry
    - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
      # Install build dependencies
    - apk update && apk add --no-cache bash curl git jq make perl
    - apk --no-cache add --virtual builds-deps build-base python
  #  - npm rebuild bcrypt --build-from-source
      # To add qemu and define IMAGE_ID
    - chmod +x .gitlab/load_env.sh
    - set -eu
    - . .gitlab/load_env.sh
      # Build the image
    - docker build -f docker/Dockerfile 
      -t ${IMAGE_ID} --build-arg target=$TARGET 
      --build-arg BUILD_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") 
      --build-arg VERSION=$VERSION 
      .
      # Push the image
    - docker push ${IMAGE_ID}

Errors in the tests:

npm install

> bcrypt@3.0.6 install /builds/*****/Gladys/server/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.6/bcrypt_lib-v3.0.6-node-v72-linux-x64-glibc.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for bcrypt@3.0.6 and node@12.13.1 (node-v72 ABI, glibc) (falling back to source compile with node-gyp) 
make: Entering directory '/builds/*****/Gladys/server/node_modules/bcrypt/build'
  CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
  SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
  COPY Release/bcrypt_lib.node
  COPY /builds/*****/Gladys/server/node_modules/bcrypt/lib/binding/bcrypt_lib.node
  TOUCH Release/obj.target/action_after_build.stamp
make: Leaving directory '/builds/*****/Gladys/server/node_modules/bcrypt/build'

> node-webcrypto-ossl@1.0.48 install /builds/*****/Gladys/server/node_modules/node-webcrypto-ossl
> node-gyp rebuild

make: Entering directory '/builds/*****/Gladys/server/node_modules/node-webcrypto-ossl/build'
  CXX(target) Release/obj.target/nodessl/src/main.o
  CXX(target) Release/obj.target/nodessl/src/core/key_exp.o
  CXX(target) Release/obj.target/nodessl/src/core/scoped_ssl.o
  CXX(target) Release/obj.target/nodessl/src/core/logger.o
  CXX(target) Release/obj.target/nodessl/src/core/excep.o
  CXX(target) Release/obj.target/nodessl/src/core/digest.o
  CXX(target) Release/obj.target/nodessl/src/core/bn.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_gen.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_jwk.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_pkcs1.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_oaep.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_pss.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_gen.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_dsa.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_dh.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_jwk.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_gen.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_ecb.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_cbc.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_gcm.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_ctr.o
  CXX(target) Release/obj.target/nodessl/src/hmac/hmac_gen.o
  CXX(target) Release/obj.target/nodessl/src/hmac/hmac_sign.o
  CXX(target) Release/obj.target/nodessl/src/pbkdf2/pbkdf2_derive.o
  CXX(target) Release/obj.target/nodessl/src/node/common.o
  CXX(target) Release/obj.target/nodessl/src/node/w_key.o
  CXX(target) Release/obj.target/nodessl/src/node/w_aes.o
  CXX(target) Release/obj.target/nodessl/src/node/w_hmac.o
  CXX(target) Release/obj.target/nodessl/src/node/w_pbkdf2.o
  CXX(target) Release/obj.target/nodessl/src/node/w_core.o
  CXX(target) Release/obj.target/nodessl/src/node/async_rsa.o
  CXX(target) Release/obj.target/nodessl/src/node/async_ec.o
  CXX(target) Release/obj.target/nodessl/src/node/async_aes.o
  CXX(target) Release/obj.target/nodessl/src/node/async_hmac.o
  CXX(target) Release/obj.target/nodessl/src/node/async_pbkdf2.o
  CXX(target) Release/obj.target/nodessl/src/node/async_core.o
  SOLINK_MODULE(target) Release/obj.target/nodessl.node
  COPY Release/nodessl.node
make: Leaving directory '/builds/*****/Gladys/server/node_modules/node-webcrypto-ossl/build'

> sqlite3@4.1.0 install /builds/*****/Gladys/server/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
[sqlite3] Success: "/builds/*****/Gladys/server/node_modules/sqlite3/lib/binding/node-v72-linux-x64/node_sqlite3.node" is installed via remote

> core-js@2.6.10 postinstall /builds/*****/Gladys/server/node_modules/core-js
> node postinstall || echo "ignore"


> nodemon@1.19.4 postinstall /builds/*****/Gladys/server/node_modules/nodemon
> node bin/postinstall || exit 0

Love nodemon? You can now support the project via the open collective:
 > https://opencollective.com/nodemon/donate

npm WARN lifecycle gladys-server@~postinstall: cannot run in wd gladys-server@ node ./cli/install_service_dependencies.js (wd=/builds/*****/Gladys/server)
npm WARN gladys-server@ No description
npm WARN gladys-server@ No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 1188 packages from 1428 contributors and audited 5335 packages in 51.181s
found 13 vulnerabilities (1 moderate, 11 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details
$ npm run prettier-check

> gladys-server@ prettier-check /builds/*****/Gladys/server
> prettier --check '**/*.js' '**/*.json'

Checking formatting...
All matched files use Prettier code style!
$ npm run eslint

> gladys-server@ eslint /builds/*****/Gladys/server
> eslint .


/builds/*****/Gladys/server/services/mqtt/index.js
  6:24  error  Unable to resolve path to module 'mqtt'  import/no-unresolved

/builds/*****/Gladys/server/services/philips-hue/index.js
  8:29  error  Unable to resolve path to module 'node-hue-api'  import/no-unresolved

/builds/*****/Gladys/server/services/philips-hue/lib/light/index.js
  1:28  error  Unable to resolve path to module 'bottleneck/es5'  import/no-unresolved

/builds/*****/Gladys/server/services/rtsp-camera/index.js
  6:26  error  Unable to resolve path to module 'fluent-ffmpeg'  import/no-unresolved

/builds/*****/Gladys/server/services/telegram/index.js
  10:31  error  Unable to resolve path to module 'node-telegram-bot-api'  import/no-unresolved

/builds/*****/Gladys/server/services/usb/api/usb.controller.js
  13:17  warning  Unexpected unnamed function  func-names

/builds/*****/Gladys/server/services/usb/index.js
  5:30  error  Unable to resolve path to module 'serialport'  import/no-unresolved

/builds/*****/Gladys/server/services/zwave/index.js
  7:25  error  Unable to resolve path to module 'openzwave-shared'  import/no-unresolved

/builds/*****/Gladys/server/test/benchmark/triggers.js
  68:8  warning  Unexpected constant condition  no-constant-condition

/builds/*****/Gladys/server/test/lib/gateway/GladysGatewayClientMock.test.js
  4:33  warning  Unexpected unnamed function  func-names

✖ 10 problems (7 errors, 3 warnings)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! gladys-server@ eslint: `eslint .`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the gladys-server@ eslint script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2019-12-05T14_54_43_396Z-debug.log

Build errors started here:

> bcrypt@3.0.6 install /src/server/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.6/bcrypt_lib-v3.0.6-node-v72-linux-x64-musl.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for bcrypt@3.0.6 and node@12.13.1 (node-v72 ABI, musl) (falling back to source compile with node-gyp) 
make: Entering directory '/src/server/node_modules/bcrypt/build'
  CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
In file included from ../src/bcrypt_node.cc:1:
../../nan/nan.h: In function 'void Nan::AsyncQueueWorker(Nan::AsyncWorker*)':
../../nan/nan.h:2232:62: warning: cast between incompatible function types from 'void (*)(uv_work_t*)' {aka 'void (*)(uv_work_s*)'} to 'uv_after_work_cb' {aka 'void (*)(uv_work_s*, int)'} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../../nan/nan.h:53,
                 from ../src/bcrypt_node.cc:1:
../src/bcrypt_node.cc: At global scope:
/root/.cache/node-gyp/12.13.1/include/node/node.h:566:43: warning: cast between incompatible function types from 'void (*)(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)' {aka 'void (*)(v8::Local<v8::Object>)'} to 'node::addon_register_func' {aka 'void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)'} [-Wcast-function-type]
       (node::addon_register_func) (regfunc),                          \
                                           ^
/root/.cache/node-gyp/12.13.1/include/node/node.h:600:3: note: in expansion of macro 'NODE_MODULE_X'
   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
   ^~~~~~~~~~~~~
../src/bcrypt_node.cc:378:1: note: in expansion of macro 'NODE_MODULE'
 NODE_MODULE(bcrypt_lib, init);
 ^~~~~~~~~~~
In file included from /root/.cache/node-gyp/12.13.1/include/node/node.h:63,
                 from ../../nan/nan.h:53,
                 from ../src/bcrypt_node.cc:1:
/root/.cache/node-gyp/12.13.1/include/node/v8.h: In instantiation of 'void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]':
/root/.cache/node-gyp/12.13.1/include/node/node_object_wrap.h:84:78:   required from here
/root/.cache/node-gyp/12.13.1/include/node/v8.h:10004:16: warning: cast between incompatible function types from 'v8::WeakCallbackInfo<node::ObjectWrap>::Callback' {aka 'void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)'} to 'Callback' {aka 'void (*)(const v8::WeakCallbackInfo<void>&)'} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/.cache/node-gyp/12.13.1/include/node/v8.h: In instantiation of 'void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]':
../../nan/nan_object_wrap.h:65:61:   required from here
/root/.cache/node-gyp/12.13.1/include/node/v8.h:10004:16: warning: cast between incompatible function types from 'v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback' {aka 'void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)'} to 'Callback' {aka 'void (*)(const v8::WeakCallbackInfo<void>&)'} [-Wcast-function-type]
  SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
  COPY Release/bcrypt_lib.node
  COPY /src/server/node_modules/bcrypt/lib/binding/bcrypt_lib.node
  TOUCH Release/obj.target/action_after_build.stamp
make: Leaving directory '/src/server/node_modules/bcrypt/build'

> node-webcrypto-ossl@1.0.48 install /src/server/node_modules/node-webcrypto-ossl
> node-gyp rebuild

make: Entering directory '/src/server/node_modules/node-webcrypto-ossl/build'
  CXX(target) Release/obj.target/nodessl/src/main.o

More logs available if needed :sweat_smile:

2

Based on your logs, I don’t think there are any errors with Bcrypt.
It starts the compilation and the messages only indicate warnings, as far as I can see (it’s not easy to read in the posts).

I think the errors are rather due to the fact that it cannot find the modules.
Try doing as in the circleci configuration, by putting on the same line:
cd server && npm install

3

I tried, but it doesn’t change anything, GitLab works a bit differently in this regard, each instruction is executed in sequence without resetting the shell.
The test task:

test_server:
  image: node:12
  stage: test
  script:
      # Install system dependencies 
    - apt-get update && apt-get install -y libudev-dev sqlite3 openssl openzwave libopenzwave1.5-dev build-essential python
      # Install global dependencies
    - npm install typescript node-gyp npm@latest -g
      # Install NPM dependencies
    - cd server && npm install
    - ls -lA
    - ls  node_modules
      # Run prettier
    - npm run prettier-check
      # Run eslint
    - npm run eslint
      # Run tests with coverage
    - npm run coverage
  coverage: /^Statements\s*:\s*([^%]+)/
  allow_failure: true
```The "complete" log:
```sh
$ npm install typescript node-gyp npm@latest -g
/usr/local/bin/node-gyp -> /usr/local/lib/node_modules/node-gyp/bin/node-gyp.js
/usr/local/bin/npm -> /usr/local/lib/node_modules/npm/bin/npm-cli.js
/usr/local/bin/npx -> /usr/local/lib/node_modules/npm/bin/npx-cli.js
/usr/local/bin/tsserver -> /usr/local/lib/node_modules/typescript/bin/tsserver
/usr/local/bin/tsc -> /usr/local/lib/node_modules/typescript/bin/tsc
+ node-gyp@6.0.1
+ npm@6.13.2
+ typescript@3.7.3
added 102 packages from 68 contributors and updated 8 packages in 312.307s
$ cd server && npm install

> bcrypt@3.0.6 install /builds/********/Gladys/server/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.6/bcrypt_lib-v3.0.6-node-v72-linux-x64-glibc.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for bcrypt@3.0.6 and node@12.13.1 (node-v72 ABI, glibc) (falling back to source compile with node-gyp) 
make: Entering directory '/builds/********/Gladys/server/node_modules/bcrypt/build'
  CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
  SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
  COPY Release/bcrypt_lib.node
  COPY /builds/********/Gladys/server/node_modules/bcrypt/lib/binding/bcrypt_lib.node
  TOUCH Release/obj.target/action_after_build.stamp
make: Leaving directory '/builds/********/Gladys/server/node_modules/bcrypt/build'

> node-webcrypto-ossl@1.0.48 install /builds/********/Gladys/server/node_modules/node-webcrypto-ossl
> node-gyp rebuild

make: Entering directory '/builds/********/Gladys/server/node_modules/node-webcrypto-ossl/build'
  CXX(target) Release/obj.target/nodessl/src/main.o
  CXX(target) Release/obj.target/nodessl/src/core/key_exp.o
  CXX(target) Release/obj.target/nodessl/src/core/scoped_ssl.o
  CXX(target) Release/obj.target/nodessl/src/core/logger.o
  CXX(target) Release/obj.target/nodessl/src/core/excep.o
  CXX(target) Release/obj.target/nodessl/src/core/digest.o
  CXX(target) Release/obj.target/nodessl/src/core/bn.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_gen.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_jwk.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_pkcs1.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_oaep.o
  CXX(target) Release/obj.target/nodessl/src/rsa/rsa_pss.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_gen.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_dsa.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_dh.o
  CXX(target) Release/obj.target/nodessl/src/ec/ec_jwk.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_gen.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_ecb.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_cbc.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_gcm.o
  CXX(target) Release/obj.target/nodessl/src/aes/aes_ctr.o
  CXX(target) Release/obj.target/nodessl/src/hmac/hmac_gen.o
  CXX(target) Release/obj.target/nodessl/src/hmac/hmac_sign.o
  CXX(target) Release/obj.target/nodessl/src/pbkdf2/pbkdf2_derive.o
  CXX(target) Release/obj.target/nodessl/src/node/common.o
  CXX(target) Release/obj.target/nodessl/src/node/w_key.o
  CXX(target) Release/obj.target/nodessl/src/node/w_aes.o
  CXX(target) Release/obj.target/nodessl/src/node/w_hmac.o
  CXX(target) Release/obj.target/nodessl/src/node/w_pbkdf2.o
  CXX(target) Release/obj.target/nodessl/src/node/w_core.o
  CXX(target) Release/obj.target/nodessl/src/node/async_rsa.o
  CXX(target) Release/obj.target/nodessl/src/node/async_ec.o
  CXX(target) Release/obj.target/nodessl/src/node/async_aes.o
  CXX(target) Release/obj.target/nodessl/src/node/async_hmac.o
  CXX(target) Release/obj.target/nodessl/src/node/async_pbkdf2.o
  CXX(target) Release/obj.target/nodessl/src/node/async_core.o
  SOLINK_MODULE(target) Release/obj.target/nodessl.node
  COPY Release/nodessl.node
make: Leaving directory '/builds/********/Gladys/server/node_modules/node-webcrypto-ossl/build'

> sqlite3@4.1.0 install /builds/********/Gladys/server/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
[sqlite3] Success: "/builds/********/Gladys/server/node_modules/sqlite3/lib/binding/node-v72-linux-x64/node_sqlite3.node" is installed via remote

> core-js@2.6.10 postinstall /builds/********/Gladys/server/node_modules/core-js
> node postinstall || echo "ignore"


> nodemon@1.19.4 postinstall /builds/********/Gladys/server/node_modules/nodemon
> node bin/postinstall || exit 0

Love nodemon? You can now support the project via the open collective:
 > https://opencollective.com/nodemon/donate

npm WARN lifecycle gladys-server@~postinstall: cannot run in wd gladys-server@ node ./cli/install_service_dependencies.js (wd=/builds/********/Gladys/server)
npm WARN gladys-server@ No description
npm WARN gladys-server@ No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 1188 packages from 1428 contributors and audited 5335 packages in 65.742s
found 13 vulnerabilities (1 moderate, 11 high, 1 critical)
  run `npm audit fix` to fix them
4

Apparently, you need to install the ‹ eslint-plugin-import › module.
However, I don’t know why…

5

I was able to fix my build task issues, now the Gladys image I get starts correctly.

Regarding the test_server task, it is also progressing, I was able to pass the eslint by running npm run postinstall

Now the error Unable to resolve path to module 'something' import/no-unresolved have disappeared, and it’s the test coverage that fails at this level:

PATCH /api/v1/house/test-house
    1) "before each" hook: beforeEach for "should update a house"

    39 passing (4m)
    1 failing

    1) "before each" hook: beforeEach for "should update a house":
       Error: Timeout of 8000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
        at listOnTimeout (internal/timers.js:531:17)
        at processTimers (internal/timers.js:475:7)