gave up on Build for BuildConfig tybsdev/basvuru-arayuz (0) due to fatal error: the LastVersion(1) on build config xxx does not match the build request LastVersion(0)

OpenShift Builder POD Failed with no error message in oc logs -f pod_name output.

**oc get events on the master showed this:

Type: Warning
Reason:BuildConfigInstantiateFailed
Source: buildconfig-controller
Message: gave up on Build for BuildConfig tybsdev/basvuru-arayuz (0) due to fatal error: the LastVersion(1) on build config tybsdev/basvuru-arayuz does not match the build request LastVersion(0)

**oc describe pod said this:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
——— ——– —– —- ————- ——– —— ——-
25m 25m 1 default-scheduler Normal Scheduled Successfully assigned basvuru-arayuz-1-build to tybsrhosnode02.defence.local
<invalid> <invalid> 1 kubelet, tybsrhosnode02.defence.local spec.containers{sti-build} Normal Pulled Container image “openshift3/ose-sti-builder:v3.6.173.0.21” already present on machine
<invalid> <invalid> 1 kubelet, tybsrhosnode02.defence.local spec.containers{sti-build} Normal Created Created container
<invalid> <invalid> 1 kubelet, tybsrhosnode02.defence.local spec.containers{sti-build} Normal Started Started container

 

**oc get pods -o wide showed that the build pod was scheduled on node2

node2 showed no problems:
**[[email protected] ~]# oc describe node tybsrhosnode02.defence.local
Name: tybsrhosnode02.defence.local
Role:
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=tybsrhosnode02.defence.local
logging-infra-fluentd=true
region=primary
Annotations: volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: <none>
CreationTimestamp: Wed, 13 Sep 2017 14:16:02 +0300
Phase:
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
—- —— —————– —————— —— ——-
OutOfDisk False Fri, 26 Oct 2018 11:53:16 +0300 Wed, 10 Oct 2018 20:09:05 +0300 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Fri, 26 Oct 2018 11:53:16 +0300 Wed, 10 Oct 2018 20:09:05 +0300 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 26 Oct 2018 11:53:16 +0300 Wed, 10 Oct 2018 20:09:05 +0300 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready True Fri, 26 Oct 2018 11:53:16 +0300 Wed, 10 Oct 2018 20:08:54 +0300 KubeletReady kubelet is posting ready status
Addresses: 172.20.30.224,172.20.30.224,tybsrhosnode02.defence.local
Capacity:
cpu: 8
memory: 131865388Ki
pods: 80
Allocatable:
cpu: 6
memory: 125618988Ki
pods: 80
System Info:
Machine ID: dfeb0732c1464538abc9eab4169868cf
System UUID: 42184533-35FC-C47E-B84F-223AE30C8645
Boot ID: e9e34a15-7d8e-4cfc-b995-3871f849f1d3
Kernel Version: 3.10.0-693.2.1.el7.x86_64
OS Image: OpenShift Enterprise
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.12.6
Kubelet Version: v1.6.1+5115d708d7
Kube-Proxy Version: v1.6.1+5115d708d7
ExternalID: tybsrhosnode02.defence.local
Non-terminated Pods: (11 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
——— —- ———— ———- ————— ————-
amq broker-drainer-2-19w50 0 (0%) 0 (0%) 0 (0%) 0 (0%)
bpm-test proj1-1-h2kbs 0 (0%) 0 (0%) 0 (0%) 0 (0%)
logging logging-fluentd-tznxt 100m (1%) 100m (1%) 512Mi (0%) 512Mi (0%)
process-server a1501-bpm-app-postgresql-4-bsrpv 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sso sso-postgresql-1-gqxfr 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev bilgi-edinme-yonetimi-1-km0dh 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev infra-test-1-c3p9q 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev komite-yonetimi-arayuz-1-pb11l 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev panel-yonetimi-arayuz-1-393cr 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev program-cagri-yonetimi-arayuz-1-x7q16 0 (0%) 0 (0%) 0 (0%) 0 (0%)
tybsdev teydeb-1-jt82x 0 (0%) 0 (0%) 0 (0%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
———— ———- ————— ————-
100m (1%) 100m (1%) 512Mi (0%) 512Mi (0%)
Events: <none>

 

SSHed into node2
Tried to fetch the builder image manually from node2
**docker pull docker-registry.default.svc:5000/openshift3/ose-sti-builder:v3.6.173.0.21

it said:
Trying to pull repository docker-registry.default.svc:5000/openshift3/ose-sti-builder …
unable to retrieve auth token: 401 unauthorized

 

Tried to pull the application’s image
**docker pull docker-registry.default.svc:5000/tybsdev/basvuru-arayuz

it said:
Using default tag: latest
Trying to pull repository docker-registry.default.svc:5000/tybsdev/basvuru-arayuz …
unable to retrieve auth token: 401 unauthorized

I loggedin to the registry:
**docker login -u admin -p $(oc whoami -t) docker-registry.default.svc:5000

Tried to pull the image again:
**docker pull docker-registry.default.svc:5000/tybsdev/basvuru-arayuz

It said:
Using default tag: latest
Trying to pull repository docker-registry.default.svc:5000/tybsdev/basvuru-arayuz …
manifest unknown: manifest unknown

Did the same for builder image:

** docker pull docker-registry.default.svc:5000/openshift3/ose-sti-builder:v3.6.173.0.21

it said:
Trying to pull repository docker-registry.default.svc:5000/openshift3/ose-sti-builder …
manifest unknown: manifest unknown

 

 

Deleted the failed pod and re-run the paremeterized jenkins pipeline for deploying this application.
Failed again.
** oc get events still displays the original problem.

 

Exported build configuration. Contents are below:

** oc export bc basvuru-arayuz
apiVersion: v1
kind: BuildConfig
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:”v1″,”kind”:”BuildConfig”,”metadata”:{“annotations”:{“openshift.io/generated-by”:”OpenShiftNewApp”},”creationTimestamp”:null,”labels”:{“app”:”basvuru-arayuz”,”template”:”tybs-s2i-newapp-template”},”name”:”basvuru-arayuz”,”namespace”:”tybsdev”},”spec”:{“nodeSelector”:null,”output”:{“to”:{“kind”:”ImageStreamTag”,”name”:”basvuru-arayuz:latest”}},”postCommit”:{},”resources”:{},”source”:{“contextDir”:”dev”,”git”:{“ref”:”develop”,”uri”:”http://serdar.onur:[email protected]:7990/scm/tybs/tybs_code.git&#8221;},”type”:”Git”},”strategy”:{“sourceStrategy”:{“env”:[{“name”:”ARTIFACT_COPY_ARGS”,”value”:”*-SNAPSHOT.jar”},{“name”:”ARTIFACT_DIR”,”value”:”basvuru/basvuru-arayuz/target/”},{“name”:”MAVEN_ARGS”,”value”:”package -Dfabric8.skip=true -Ddb=postgres -DskipTests=true -pl basvuru/basvuru-arayuz –also-make”},{“name”:”MAVEN_MIRROR_URL”,”value”:”http://192.168.63.121:8081/repository/maven-public/&#8221;}],”from”:{“kind”:”ImageStreamTag”,”name”:”fis-java-openshift:latest”,”namespace”:”openshift”}},”type”:”Source”},”triggers”:[{“github”:{“secret”:”wjj3wH0ppJ9nNc4lQC6_”},”type”:”GitHub”},{“generic”:{“secret”:”jZEi04f0yjPpkDYbaxR4″},”type”:”Generic”},{“type”:”ConfigChange”},{“imageChange”:{},”type”:”ImageChange”}]},”status”:{“lastVersion”:0}}
openshift.io/generated-by: OpenShiftNewApp
creationTimestamp: null
labels:
app: basvuru-arayuz
template: tybs-s2i-newapp-template
name: basvuru-arayuz
spec:
nodeSelector: null
output:
to:
kind: ImageStreamTag
name: basvuru-arayuz:latest
postCommit: {}
resources: {}
runPolicy: Serial
source:
contextDir: dev
git:
ref: develop
uri: http://serdar.onur:[email protected]:7990/scm/tybs/tybs_code.git
type: Git
strategy:
sourceStrategy:
env:

  • name: ARTIFACT_COPY_ARGS

value: ‘*-SNAPSHOT.jar’

  • name: ARTIFACT_DIR

value: basvuru/basvuru-arayuz/target/

  • name: MAVEN_ARGS

value: package -Dfabric8.skip=true -Ddb=postgres -DskipTests=true -pl basvuru/basvuru-arayuz
–also-make

  • name: MAVEN_MIRROR_URL

value: http://192.168.63.121:8081/repository/maven-public/
from:
kind: ImageStreamTag
name: fis-java-openshift:latest
namespace: openshift
type: Source
triggers:

  • github:

secret: wjj3wH0ppJ9nNc4lQC6_
type: GitHub

  • generic:

secret: jZEi04f0yjPpkDYbaxR4
type: Generic

  • type: ConfigChange
  • imageChange: {}

type: ImageChange
status:
lastVersion: 0