Tag Archives: OpenShift

Spring Boot + OpenShift + Auto-Reconfiguration

Posted on by | 2 Replies

I wanted to know how easy it would be to get Spring-Boot auto-reconfiguration to work in OpenShift. It did not really work for me in Cloud Foundry without some massaging ( auto re-config blog ).

The proper way would be to extend https://github.com/spring-projects/spring-cloud/tree/master/spring-cloud-core for OpenShift Which is probably dead simple, too, but I wanted to do a quick 30 minute hack which become about a 90 minutes dirty hack.

In simple terms, the cartridge always sets -Dspring.profiles.active=cloud (unless this is overwritten by the app developer). And when it finds a ‘OPENSHIFT_MYSQL_DB_URL’ environment variable (which means the user added that service (cartridge) to the application, then the cartridge will add a the MariaDB (MySQL) information to ‘VCAP_SERVICES’ In addition I als set ‘VCAP_APPLICATION’ to some partially correct values, as well as VCAP_APP_HOST and VCAP_APP_PORT.

Then I deployed the jar generated from the unfamous-quotes cloud-foundry-autoreconfig-tests branch and it correctly picked up the MySQL DB and SUCCESS.

It was ridiculously easy to get that working in OpenShift. If I hadn’t put in a typo in one of my first attempts it would have been even easier.

What I did not mentioned in the last blog was that the version of the “unfamous quotes” code only works on Cloud Foundry if there is a data source supplied by Cloud Foundry. Otherwise you get:

org.springframework.cloud.CloudException: No unique service matching interface javax.sql.DataSource found. Expected 1, found 0

This happens due to the this code in DataSourceConfiguration:

return connectionFactory().dataSource();

With the current version of the spring runner cartridge the same happens: You have to add a MariaDB or it wont work.

Since this was just a quick and dirty test, I did not add any other DB services. But I also would rather check out the spring-core-cloud and create an OpenShift Cloud Connector.

OpenShift Local VM Environment Notes

Posted on by | Leave a reply

Initial Set-up

I was able to use an older version of my non-spring test server simple-executable-jar-web-server.

The test is following up on my tests with Cloud Foundry as well as Stackato.

I also created a very simple cartridge (notes) that I was using to understand the way cartridges work. But a
DIY cartridge would have been sufficent for this.

App container cartridge are supposed to come with templates as far as I understand. In this case I packaged the
test server in the template which is also the reason why I wanted it to stay small.

Runtime Environment and Properties

Environment

'OPENSHIFT_DEPLOYMENTS_DIR' = '/var/lib/openshift/539/app-deployments/'
'HISTFILE' = '/var/lib/openshift/539/app-root/data/.bash_history'
'OPENSHIFT_SBJR_JAR_DIR' = '/var/lib/openshift/539/app-root/runtime/repo//target/'
'OPENSHIFT_DATA_DIR' = '/var/lib/openshift/539/app-root/data/'
'JAVA_HOME' = '/etc/alternatives/java_sdk_1.6.0'
'OPENSHIFT_SBJR_VERSION' = '1.0.2'
'OPENSHIFT_SBJR_JPDA_PORT' = '8787'
'PWD' = '/var/lib/openshift/539/spring-boot-jar-runner'
'TMP_DIR' = '/tmp/'
'OPENSHIFT_SECRET_TOKEN' = 'n_3eZmWnvvsJUnY4nYicj-k6vVAG_6nglPrtgkqjyKf0OWH-pGlRbVyCRGRcFAruY4n8BKX7aPKSvZxO-5lQTlWheStOyHkmBcsQ4zwlmJaMjNk9Ii5IkzktC0P9seXS'
'NLSPATH' = '/usr/dt/lib/nls/msg/%L/%N.cat'
'OPENSHIFT_APP_DNS' = 'app1-testapps.openshift.local'
'TMPDIR' = '/tmp/'
'TMP' = '/tmp/'
'OPENSHIFT_SBJR_LOG_DIR' = '/var/lib/openshift/539/spring-boot-jar-runner//logs/'
'OPENSHIFT_NAMESPACE' = 'testapps'
'PATH' = '/etc/alternatives/java_sdk_1.6.0/bin:/bin:/usr/bin:/usr/sbin'
'OPENSHIFT_SBJR_IP' = '127.1.245.1'
'OPENSHIFT_SBJR_IDENT' = 'coder_mnm_at:spring-boot-jar-runner:1.0.2:0.0.1'
'OPENSHIFT_UMASK' = '022'
'OPENSHIFT_APP_UUID' = '539'
'OPENSHIFT_CARTRIDGE_SDK_BASH' = '/usr/lib/openshift/cartridge_sdk/bash/sdk'
'OPENSHIFT_BUILD_DEPENDENCIES_DIR' = '/var/lib/openshift/539/app-root/runtime/build-dependencies/'
'OPENSHIFT_SBJR_MAX_STARTUP_WAIT_IN_SEC' = '60'
'OPENSHIFT_SBJR_DIR' = '/var/lib/openshift/539/spring-boot-jar-runner/'
'OPENSHIFT_GEAR_DNS' = 'app1-testapps.openshift.local'
'OPENSHIFT_REPO_DIR' = '/var/lib/openshift/539/app-root/runtime/repo/'
'OPENSHIFT_BROKER_HOST' = 'localhost'
'SHLVL' = '3'
'OPENSHIFT_GEAR_MEMORY_MB' = '512'
'XFILESEARCHPATH' = '/usr/dt/app-defaults/%L/Dt'
'OPENSHIFT_APP_NAME' = 'app1'
'OPENSHIFT_GEAR_UUID' = '539'
'OPENSHIFT_SBJR_JAR_RELATIVE_FOLDER_NAME' = 'target'
'OPENSHIFT_APP_SSH_PUBLIC_KEY' = '/var/lib/openshift/539/.openshift_ssh/id_rsa.pub'
'OPENSHIFT_GEAR_NAME' = 'app1'
'LOGNAME' = '539'
'OPENSHIFT_SBJR_JAR_BASE_DIR' = '/var/lib/openshift/539/app-root/runtime/repo/'
'OPENSHIFT_PRIMARY_CARTRIDGE_DIR' = '/var/lib/openshift/539/spring-boot-jar-runner/'
'_' = '/bin/java'
'OPENSHIFT_SBJR_HTTP_PORT' = '8080'
'OPENSHIFT_CLOUD_DOMAIN' = 'openshift.local'
'OPENSHIFT_DEPENDENCIES_DIR' = '/var/lib/openshift/539/app-root/runtime/dependencies/'
'SHELL' = '/bin/sh'
'Copy of OPENSHIFT_SBJR_MAX_SHUTDOWN_WAIT_IN_SEC' = '60'
'OPENSHIFT_SBJR_LOG' = '/var/lib/openshift/539/spring-boot-jar-runner//logs/main-SBJR-log'
'OPENSHIFT_HOMEDIR' = '/var/lib/openshift/539/'
'GEM_HOME' = '/var/lib/openshift/539/.gem'
'OPENSHIFT_TMP_DIR' = '/tmp/'
'OPENSHIFT_APP_SSH_KEY' = '/var/lib/openshift/539/.openshift_ssh/id_rsa'
'OPENSHIFT_CARTRIDGE_SDK_RUBY' = '/usr/lib/openshift/cartridge_sdk/ruby/sdk.rb'
'USER' = '539'
'HOME' = '/var/lib/openshift/539'
'OPENSHIFT_SBJR_JDK6' = '/etc/alternatives/java_sdk_1.6.0'
'OPENSHIFT_SBJR_JDK7' = '/etc/alternatives/java_sdk_1.7.0'
'OPENSHIFT_SBJR_PATH_ELEMENT' = '/etc/alternatives/java_sdk_1.6.0/bin'

System Properties

'java.runtime.name' = 'OpenJDK Runtime Environment'
'sun.boot.library.path' = '/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/amd64'
'java.vm.version' = '24.45-b08'
'java.vm.vendor' = 'Oracle Corporation'
'java.vendor.url' = 'http://java.oracle.com/'
'path.separator' = ':'
'java.vm.name' = 'OpenJDK 64-Bit Server VM'
'file.encoding.pkg' = 'sun.io'
'user.country' = 'US'
'sun.java.launcher' = 'SUN_STANDARD'
'sun.os.patch.level' = 'unknown'
'server.port' = ''
'java.vm.specification.name' = 'Java Virtual Machine Specification'
'user.dir' = '/var/lib/openshift/539/spring-boot-jar-runner'
'java.runtime.version' = '1.7.0_45-mockbuild_2013_10_16_17_47-b00'
'java.awt.graphicsenv' = 'sun.awt.X11GraphicsEnvironment'
'java.endorsed.dirs' = '/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/endorsed'
'os.arch' = 'amd64'
'java.io.tmpdir' = '/tmp'
'line.separator' = '
'
'java.vm.specification.vendor' = 'Oracle Corporation'
'os.name' = 'Linux'
'sun.jnu.encoding' = 'ANSI_X3.4-1968'
'java.library.path' = '/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib'
'java.specification.name' = 'Java Platform API Specification'
'java.class.version' = '51.0'
'sun.management.compiler' = 'HotSpot 64-Bit Tiered Compilers'
'os.version' = '3.11.10-200.fc19.x86_64'
'user.home' = '/var/lib/openshift/539'
'user.timezone' = 'US/Pacific'
'java.awt.printerjob' = 'sun.print.PSPrinterJob'
'file.encoding' = 'ANSI_X3.4-1968'
'java.specification.version' = '1.7'
'server.address' = '127.1.245.1'
'java.class.path' = '/var/lib/openshift/539/spring-boot-jar-runner/web-app-jar/simple-executable-jar-web-server-0.0.1.jar'
'user.name' = '539'
'java.vm.specification.version' = '1.7'
'sun.java.command' = '/var/lib/openshift/539/spring-boot-jar-runner/web-app-jar/simple-executable-jar-web-server-0.0.1.jar'
'java.home' = '/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre'
'sun.arch.data.model' = '64'
'user.language' = 'en'
'java.specification.vendor' = 'Oracle Corporation'
'awt.toolkit' = 'sun.awt.X11.XToolkit'
'java.vm.info' = 'mixed mode'
'logging.path' = '/var/lib/openshift/539/spring-boot-jar-runner//logs/'
'java.version' = '1.7.0_45'
'java.ext.dirs' = '/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/ext:/usr/java/packages/lib/ext'
'sun.boot.class.path' = '/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/netx.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/plugin.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/rhino.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc19.x86_64/jre/classes'
'java.vendor' = 'Oracle Corporation'
'file.separator' = '/'
'java.vendor.url.bug' = 'http://bugreport.sun.com/bugreport/'
'sun.io.unicode.encoding' = 'UnicodeLittle'
'sun.cpu.endian' = 'little'
'sun.cpu.isalist' = ''
Here is what I know about you :
Your adress:
/127.0.0.1:23554
Your request headers:
'X-forwarded-proto' = '[http]'
'X-request-start' = '[t=1402811035856892]'
'Host' = '[app1-testapps.openshift.local]'
'X-forwarded-server' = '[broker-66587a.openshift.local]'
'Accept-encoding' = '[gzip,deflate,sdch]'
'X-forwarded-port' = '[80]'
'Connection' = '[close]'
'X-forwarded-for' = '[192.168.121.33]'
'X-client-ip' = '[192.168.121.33]'
'Accept-language' = '[en-GB,en-US;q=0.8,en;q=0.6,de;q=0.4]'
'X-forwarded-host' = '[app1-testapps.openshift.local]'
'User-agent' = '[Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36]'
'Accept' = '[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8]'

Adding MariaDB

From the three experiences I found the OpenShift one the friendliest in terms of adding a new service as well as the information that I got out of it.
It also immediately showed the connection details to the DB.
Only a restart is required, not a full push.

MySQL 5.1 database added.  Please make note of these credentials:

    Root User: admin123
Root Password: passwd123
Database Name: app1

Connection URL: mysql://$OPENSHIFT_MARIADB_DB_HOST:$OPENSHIFT_MARIADB_DB_PORT/

You can manage your new MySQL database by also embedding phpmyadmin.
The phpmyadmin username and password will be the same as the MySQL credentials above.

Yep, it is MySQL.

A lot more variables changed in comparison to the other two and no system properties:

> 'OPENSHIFT_MYSQL_DB_HOST' = '127.1.245.2'
> 'OPENSHIFT_MARIADB_DB_PORT' = '3306'
> 'OPENSHIFT_MYSQL_DB_PORT' = '3306'
> 'OPENSHIFT_MARIADB_DB_PASSWORD' = 'passwd123'
> 'OPENSHIFT_MARIADB_DIR' = '/var/lib/openshift/539/mariadb/'
> 'OPENSHIFT_MARIADB_DB_USERNAME' = 'admin123'
> 'OPENSHIFT_MARIADB_DB_SOCKET' = '/var/lib/openshift/539/mariadb//socket/mysql.sock'
> 'OPENSHIFT_MYSQL_DB_URL' = 'mysql://admin123:passwd123@127.1.245.2:3306/'
> 'OPENSHIFT_MARIADB_DB_HOST' = '127.1.245.2'
> 'OPENSHIFT_MYSQL_DB_USERNAME' = 'admin123'
> 'OPENSHIFT_MYSQL_DB_LOG_DIR' = '/var/lib/openshift/539/mariadb//log/'
> 'OPENSHIFT_MARIADB_IDENT' = 'redhat:mariadb:5.5:0.2.4'
> 'OPENSHIFT_MARIADB_DB_LOG_DIR' = '/var/lib/openshift/539/mariadb//log/'
> 'OPENSHIFT_MARIADB_DB_URL' = 'mysql://admin123:passwd123@127.1.245.2:3306/'
> 'OPENSHIFT_MYSQL_DB_SOCKET' = '/var/lib/openshift/539/mariadb//socket/mysql.sock'
> 'OPENSHIFT_MYSQL_DB_PASSWORD' = 'passwd123'

Similar to Stackato there is a lot of duplication, in both cases I assume the reason is backward compatibility.

Adding MongoDB and PostgreSQL as well

MongoDB 2.2 database added.  Please make note of these credentials:

Root User:     adminuser
Root Password: passwd123
Database Name: app1

Connection URL: mongodb://$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/



PostgreSQL 9.2 database added.  Please make note of these credentials:

    Root User: adminuser
    Root Password: passwd123
    Database Name: app1

Connection URL: postgresql://$OPENSHIFT_POSTGRESQL_DB_HOST:$OPENSHIFT_POSTGRESQL_DB_PORT

changes are :

> 'LD_LIBRARY_PATH' = ''
> 'MANPATH' = ''
> 'OPENSHIFT_MONGODB_DB_HOST' = '127.1.245.3'
> 'OPENSHIFT_MONGODB_DB_LOG_DIR' = '/var/lib/openshift/539/mongodb//log/'
> 'OPENSHIFT_MONGODB_DB_PASSWORD' = 'passwd123'
> 'OPENSHIFT_MONGODB_DB_PORT' = '27017'
> 'OPENSHIFT_MONGODB_DB_URL' = 'mongodb://adminuser:passwd123@127.1.245.3:27017/'
> 'OPENSHIFT_MONGODB_DB_USERNAME' = 'adminuser'
> 'OPENSHIFT_MONGODB_DIR' = '/var/lib/openshift/539/mongodb/'
> 'OPENSHIFT_MONGODB_IDENT' = 'redhat:mongodb:2.2:0.2.5'
> 'OPENSHIFT_POSTGRESQL_DB_HOST' = '127.1.245.4'
> 'OPENSHIFT_POSTGRESQL_DB_LOG_DIR' = '/var/lib/openshift/539/postgresql//log/'
> 'OPENSHIFT_POSTGRESQL_DB_PASSWORD' = 'passwd123'
> 'OPENSHIFT_POSTGRESQL_DB_PID' = '/var/lib/openshift/539/postgresql//pid/postgres.pid'
> 'OPENSHIFT_POSTGRESQL_DB_PORT' = '5432'
> 'OPENSHIFT_POSTGRESQL_DB_SOCKET' = '/var/lib/openshift/539/postgresql//socket/'
> 'OPENSHIFT_POSTGRESQL_DB_URL' = 'postgresql://adminuser:passwd123@127.1.245.4:5432'
> 'OPENSHIFT_POSTGRESQL_DB_USERNAME' = 'adminuser'
> 'OPENSHIFT_POSTGRESQL_DIR' = '/var/lib/openshift/539/postgresql/'
> 'OPENSHIFT_POSTGRESQL_IDENT' = 'redhat:postgresql:9.2:0.3.5'
> 'OPENSHIFT_POSTGRESQL_PATH_ELEMENT' = '/bin'
> 'OPENSHIFT_POSTGRESQL_VERSION' = '9.2'
> 'PGDATABASE' = 'app1'
> 'PGDATA' = '/var/lib/openshift/539/postgresql//data'
> 'PGHOST' = '/var/lib/openshift/539/postgresql//socket/'
> 'PGUSER' = 'adminuser'

Building an OpenShift Cartridge

Posted on by | Leave a reply

The Basics

I used the DIY cartridge before and played with it a bit. Next I wanted to create a custom cartridge. As usual for me the first attempt was more about learning about the system than about getting things to work right away. And I must say it was a bit annoying when you start with zero or little knowledge.

A working version is here : https://bitbucket.org/markus_ebenhoeh/openshift-cartridge-spring-boot-jar-runner

Learn from my mistakes:

  • make sure that you use the right cartridge version. I followed code that was newer than the version I was running, but even then I mixed old with new conventions. E.g. I used OPENSHIFT_LOG_DIR but that does not exist yet.

  • don’t git push your code, then find the manifest URL and then try to test it. Either use the card reflector (usage) or if you already have an http server running, just serve your code directly I changed the source to a zip on my http server and ran this just before I retried with a new application rm -f cartridge-name.zip; zip -r cartridge-name.zip *; chmod a+rX -R $PWD In the end, I just mounted a shared folder and referred the source URL to that with file:///mnt/public-shared/openshift-cartridge-spring-boot-jar-runner while still hosting the same folder via http for the manifest access

  • to test my cartridge I added a spring-boot application jar with an embedded tomcat. Instead of taking the smallest web-app jar that I could find I took a 29MB jar and added it into the template. This is stupid on many levels, but it also meant that it took long to do the git pull and the start and the transfer.etc having a test app in the template is important for the testing of the cartridge and afterwards for the use of it, but a smaller application is what I am going for next.

Debugging a Failing Cartridge

When my cartridge completely failed I had no output and nothing to go on. I was logged on to my test origin VM but the gear/application folder would simple be deleted at the end. So all my test log output that was supposed to show me were it failed and what the environment looked like would vanish

So while being logged on to the test VM with root I did this dodgy thing:

cd /var/lib/openshift/
# my user/gear/app folder  always starts with 5 and I had no other gears set-up
unalias cp
while true; do cp -a 5* tmp/ ;  usleep 100000; echo next  ;done

once it failed I had more information to go on with. especially the nev folders are useful

Other problems:

  • I would like to see a cartridge life-cycle flow. e.g. copy file, call install, call setup etc.
  • when are ENV variables available? Can I refer to my own env_xyz.erb variables when defining new ones?
  • I still have no idea how the repo
  • there has to be a better way to test this then to do an add-app with the updated cartridge.
  • I could not find any debug output when my cartridge completely failed
  • CDK is not working on Origin bug

OpenShift Cartridge Development Kit

ran this against OpenShift Online since CDK is not working on Origin bug. not sure what the point is, though. I think it is about being able to have a simple way of hosting the cartridge for a certain version

rhc create-app mycart http://cdk-claytondev.rhcloud.com
The cartridge 'http://cdk-claytondev.rhcloud.com' will be downloaded and installed

Application Options
-------------------
Domain:     testcdk
Cartridges: http://cdk-claytondev.rhcloud.com
Gear Size:  default
Scaling:    no

Creating application 'mycart' ... done

The CDK is configured with password XXXXXXXXXXXXXXX for builds (use 'admin' as the user)

Waiting for your DNS name to be available ... done

Cloning into 'mycart'...
The authenticity of host 'mycart-testcdk.rhcloud.com (54.204.113.11)' can't be established.
RSA key fingerprint is cf:ee:77:cb:0e:fc:02:d7:72:7e:ae:80:c0:90:88:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mycart-testcdk.rhcloud.com,54.204.113.11' (RSA) to the list of known hosts.
X11 forwarding request failed on channel 0

Your application 'mycart' is now available.

URL:        http://mycart-testcdk.rhcloud.com/
SSH to:     539@mycart-testcdk.rhcloud.com
Git remote: ssh://539@mycart-testcdk.rhcloud.com/~/git/mycart.git/
Cloned to:  /xdata/local/dev/markus/git/cdk/mycart

Run 'rhc show-app mycart' for more details about your app.

Lifeccyle Notes

first installation

20140609_034324 setup called with --version 1.0.2 
20140609_034324 install called with --version 1.0.2 
20140609_034324 control called with start 
20140609_034324 start_app called 
20140609_034324 Starting spring-boot-bin-jar-runner cartridge 
20140609_034324 start_app: Using jarfile=/var/lib/openshift/539/spring-boot-jar-runner/web-app-jar/simple-executable-jar-web-server-0.0.1.jar 
20140609_034324 control : checkSpringBootAppIsRunning still checking 
20140609_034325 control start successfully started the application

git push new app code

20140609_034411 control called with stop 
20140609_034411 stop_app called 
20140609_034411 Stopping spring-boot-bin-jar-runner cartridge 
20140609_034411 Sending SIGTERM to java:4556 ... 
20140609_034411 control called with pre-repo-archive 
20140609_034412 control called with update-configuration 
20140609_034412 control called with pre-build 
20140609_034412 control called with build 
20140609_034412 build called with 
20140609_034412 control called with update-configuration 
20140609_034412 control called with deploy 
20140609_034412 deploy called with 
20140609_034412 control called with start 
20140609_034412 start_app called 
20140609_034412 Starting spring-boot-bin-jar-runner cartridge 
20140609_034412 start_app: Using jarfile=/var/lib/openshift/539/spring-boot-jar-runner/web-app-jar/simple-executable-jar-web-server-0.0.1.jar 
20140609_034412 control : checkSpringBootAppIsRunning still checking 
20140609_034412 control start successfully started the application 
20140609_034432 control called with post-deploy

There is considerable time spent between “start” and “post-deploy” not sure what’s happening there.