Archives de l’auteur : davidhxxx

airflow standalone pip install « apache-airflow==2.9.3 » –constraint « https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-3.11.txt » Troubleshooting Problem: The Airflow program is not available after the installation Solution: Look in the path variable to see if the airflow directory is present, if it isn’t, add it such as: :/home/david/.local/bin … Continuer la lecture →

Remove and reinstall a customized airflow chart re-install-airflow-chart.sh: cat re-install-airflow-chart.sh #!/bin/bash helm uninstall person-airflow –keep-history   kubectl delete secret person-airflow-broker-url kubectl delete secret person-airflow-fernet-key kubectl delete secret person-airflow-redis-password kubectl delete secret sh.helm.release.v1.person-airflow.v1 kubectl delete secret sh.helm.release.v1.person-airflow.v2 kubectl delete secret sh.helm.release.v1.person-airflow.v3 … Continuer la lecture →

Example to create a connection and print the exception if it fails: import traceback   import psycopg2   try: conn = psycopg2.connect( "dbname=’modeler’ user=’postgres’ host=’host.minikube.internal’ " "port=’5433′ password=’secret’, " ) print(f"conn={conn}") except Exception as e: print(traceback.format_exc())   try: conn = … Continuer la lecture →

How to print memory usage during the job execution? You can add in a script line this background processing: – while true; do echo « Memory usage in megabytes: » && echo $((`cat /sys/fs/cgroup/memory/memory.memsw.usage_in_bytes | awk ‘{print $1}’`/1048576)) && sleep 0.1; done … Continuer la lecture →

Install and basic configuration of helm – install helm: curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash – install chartmuseum: curl https://raw.githubusercontent.com/helm/chartmuseum/main/scripts/get-chartmuseum | bash – run chartmuseum: chartmuseum –debug –port=8081 \ –storage="local" \ –storage-local-rootdir="./chartstorage" & – installing charts into kubernetes: Add the URL to … Continuer la lecture →

How host its own PyPi server? There are multiple ways to host its own Python packages. The most used and documented are: – devpi-server (Python lib) – pypiserver (Python lib) – artifactory professional version (Repository manager handing many things) devpi-server … Continuer la lecture →

Map a dictionary to a custom object, or a list of dictionaries to a list of custom objects and reversely Here we refer to a dictionary but we can expand the idea to a JSON also because the JSON load … Continuer la lecture →

Examples of code Rest Controller to shutdown a spring boot application Controller import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.SpringApplication; import org.springframework.context.ApplicationContext; import org.springframework.http.HttpStatus; import org.springframework.http.MediaType; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.ResponseBody; import org.springframework.web.bind.annotation.RestController;   @RestController public class AdminController {   private … Continuer la lecture →

Enable systemd and prevent any modification on the /etc/resolv.conf to prevent any internet problem access: 2 steps: – Edit /etc/wsl.conf with: [boot] systemd=true [network] generateResolvConf = false – hardcode the content of /etc/resolv.conf: sudo rm /etc/resolv.conf sudo bash -c ‘echo … Continuer la lecture →