Category Archives: Engineering

Kubernetes as a Panacea: Myth or Reality?

In the rapidly evolving world of technology, few tools have garnered as much attention as Kubernetes. Often hailed as a silver bullet for a multitude of IT challenges, Kubernetes is touted as a panacea for managing containerized applications. But is this perception grounded in reality, or is it merely a myth? In this blog post, we will delve into the capabilities and limitations of Kubernetes, exploring whether it truly lives up to the hype.

What is Kubernetes?

Kubernetes, often abbreviated as K8s, is an open-source platform designed to automate the deployment, scaling, and operation of application containers across clusters of hosts. Originally developed by Google, it is now maintained by the Cloud Native Computing Foundation (CNCF). Kubernetes provides a framework to run distributed systems resiliently, handling scaling and failover for applications, and providing deployment patterns and management tools.

The Promises of Kubernetes

Scalability: One of the most significant promises of Kubernetes is its ability to scale applications seamlessly. It can automatically adjust the number of running containers based on the current load, ensuring that applications remain responsive and efficient under varying demands.
Resilience: Kubernetes offers robust mechanisms for managing the lifecycle of applications. It ensures that your applications are always running in the desired state, automatically restarting containers that fail or are unresponsive.
Portability: Kubernetes abstracts away the underlying infrastructure, making it possible to run your applications on any cloud provider or on-premises environment. This portability is a key advantage for businesses looking to avoid vendor lock-in.
Efficiency: By optimizing the use of resources through its scheduling capabilities, Kubernetes can lead to more efficient utilization of hardware, reducing costs and improving performance.
Automation: Kubernetes automates many aspects of application management, including deployment, scaling, and operations, freeing up developers and IT staff to focus on more strategic tasks.

The Reality Check

While the promises of Kubernetes are compelling, it’s essential to recognize that it is not a magic solution that will solve all problems effortlessly. There are several considerations and challenges that organizations must be aware of:

Complexity: Kubernetes has a steep learning curve. Its powerful features come with a complexity that can be overwhelming for teams new to container orchestration. Proper training and expertise are required to harness its full potential.
Resource Intensive: Running a Kubernetes cluster can be resource-intensive. The control plane components and various add-ons needed for a production environment can consume significant CPU and memory, which may not be ideal for small-scale applications.
Operational Overhead: Despite its automation capabilities, Kubernetes still requires significant operational oversight. Maintaining, updating, and securing a Kubernetes cluster involves ongoing effort and vigilance.
Security: Kubernetes security is complex and multi-faceted. Misconfigurations can lead to vulnerabilities, and securing a cluster requires a thorough understanding of Kubernetes security best practices and continuous monitoring.
Integration and Compatibility: Integrating Kubernetes with existing systems and workflows can be challenging. Not all applications are designed to run in a containerized environment, and some may require substantial refactoring to be compatible with Kubernetes.
Database Migrations: One of the notable limitations of Kubernetes is its inability to manage complex database migrations effectively. While Kubernetes excels at managing stateless applications, handling stateful components like databases and their intricate migration processes can be cumbersome. Database migrations often require precise sequencing, careful coordination, and rollback mechanisms that go beyond the orchestration capabilities of Kubernetes. These tasks often necessitate external tools and manual oversight, underscoring that Kubernetes is not a comprehensive solution for every aspect of application deployment and maintenance.

Use Istio

Attending recently a potential candidate meeting, the candidate was asked to “provide a system solution that would solve the most common issue found in a live-update system: uptime“. The requirements were simple. You have a system that is broken into two parts:

Backend.
Database.

Using k8s what would be the ideal setup you would choose so that you can update your Backend code, while also doing a database migration?

For example, say the backend code is compatible with the db code at major version X, what happens if you upgrade to the major version Y, and you need all your users to be serviced with 0 (literally 0), downtime?

The candidate, responded, after thinking a bit, that you need, in the between to deploy a backwards compatible version, so as to give a chance to the migration to take place.

This was unacceptable of course according to the interviewer. The answer from the interviewer was to : Use Istio.

What does this Istio thing:

Traffic Management: Imagine you have a bunch of tiny services (like small programs) that need to talk to each other to make a larger application work, much like how team members need to communicate to complete a project. Istio acts as the “traffic controller,” making sure these messages go to the right place, in the right order, and without getting lost. It can also control how much traffic goes to each service, balance the load, and handle failures gracefully.
Security: Istio makes sure that communication between services is secure. It’s like locking all the doors and windows in a building to ensure that only authorized people can enter. Istio automatically encrypts the data being sent between services and checks that each service is who it claims to be (authentication). It also enforces rules about who can talk to whom (authorization).
Observability: Istio helps you keep an eye on what’s happening inside your application. It’s like having a surveillance system that monitors traffic, performance, errors, and other important metrics. Istio provides insights into how services are interacting, which can help you quickly identify and fix issues.
Policy Enforcement: It ensures that certain rules are followed. For example, it can enforce limits on how much data one service can send to another or ensure that certain services are only accessible under specific conditions. It’s like having company policies to ensure everything runs smoothly and safely.

Ok, so my guess is that the interviewer had read the above bullet points and thought:

Hmmm, smart traffic management looks like can be used so that we can route traffic from the database while running a migration in a way that it would never stop servicing at all!

Well, that was correct. Except that the database is a central place that gathers all application traffic and requires a central semaphore to perform write applications. This main’s semaphore’s task is to allow or not allow one agent writing or non writing. That means, when you are running your migration and you are changing the columns of a table, the RDBMS needs to have exclusive access to that table in order to recreate it, and no one else is allowed to write there. No matter how smart you will route the pod logic and traffic, the end bottleneck would be the database.

Kubernetes: Panacea or Powerful Tool?

Kubernetes is undoubtedly a powerful tool that can transform how organizations deploy and manage applications. It offers unparalleled capabilities for scalability, resilience, and efficiency. However, it’s not a one-size-fits-all solution. The notion of Kubernetes as a panacea is a myth if taken to mean that it can solve all IT problems without effort or expertise.

The reality is that Kubernetes can provide immense benefits when used appropriately and with the right level of investment in knowledge and resources. Organizations should approach Kubernetes with a clear understanding of its strengths and limitations, ensuring they have the necessary skills and infrastructure in place to leverage its full potential.

In conclusion, Kubernetes is not a magical cure-all, but it is a powerful tool that, when used correctly, can significantly enhance the agility, reliability, and scalability of modern applications. By setting realistic expectations and preparing adequately, organizations can avoid the pitfalls and maximize the advantages of adopting Kubernetes.

AI, API, ChatGPT, Code, Engineering, Javascript, Prompt Engineering, ReactJS

Prompt Engineering (Not)

April 20, 2023 Dimitris Leave a comment

Hello people! I’m back with a more engineering related post this time! I will explain how I managed to create a ChatGPT app, using a couple of other API’s resulting into this:

Image created by DALL-E — Image created using DALL-E, and the phrase “not sure fry futurama”

https://myopinion.fyi

In this blog post I will attempt to walk you through the entire experience of creating a full stack mini site, using the ChatGPT API from OpenAI, and building a prompt.

My attempt will be holistic, from inception to creation, in any case this happened in the timespan of a whole week. Only the implementation was done in a weekend.

Sleep deprivation

One day I had some issues sleeping, and while staying awake, I decided to play a bit more with ChatGPT. While analysing the way it works, and also highlighting what it can do best, I had a good idea. ChatGPT, can easily summarise a lot of data in a written form. That’s because of the way it is built, the LLM, can “understand” the text input and create vectors that are weighted which in turn changes the output. In plain words, it can be fed some data and asked to create a summary of it, in a good(ish) way.

The other aspect of that idea comes from the fact that I really enjoy eating at good restaurants. And travelling to many different places. Unfortunately, when you reach a place you haven’t visited before, and you are hungry like me, you would want to eat something. So you have two options:

Ask local people, but you might get upsold something you might not like.
Go to google maps restaurants and see what’s around that is good.

Google Maps Restaurant Reviews

Google offers a nice alternative. After we open the map app, we are met with a nice overview which shows all the available restaurants in the area, along with their average overall rating.

If you click the button restaurants, you are getting a list with a lot of restaurants that are in the vicinity, but that list isn’t ordered, not representative of how good the restaurant is. That is because, you cannot sort that list based on how many reviews there are, or which rating they have.

This is something I do not like with Google Maps Restaurant Reviews. I often have to scroll down all the time and search for the ones in the area with the highest number of reviews and rating in order to decide (more on that later why I didn’t implement my app that way — which will come at some time).

Clicking through one restaurant and you get the below section:

You get the rating (4.6 / 5) , the total number of reviews and the type of restaurant (here Fusion)

It includes the rating, the number of reviews and the type of restaurant.

Sure, but, why ChatGPT is needed here?

I’m glad you asked. If you click below, at the “reviews” , you are met with a breakdown of “the most relevant” according to google, reviews for that restaurant.

In that view, among other info, you see the actual user review, with some photos and his or hers total rating (ie. 5/5 in this case).

You guessed it right!

The My Opinion app is getting 5 of the “most relevant” reviews, along with the average rating and my own flavour of personal review prompt and calls the chat gpt api.

The prompt goes like this:

Restaurant: 'hoocut' near Platia Agias Irinis 9, Athina, Overall Rating: 4.3/5 from 2827 Reviews, types: restaurant,food,point_of_interest,establishment

Juliana M, Rating: 5/5, Visited: 10 months ago
Experience: For Greece’s version of “fast food” this was high quality and tasty. Only place I found a vegetarian gyro — had grape leaves inside. It was delicious. Also had the souvlaki over fries which was amazing. Able to sit in or dine out! Quick service. Wish we had an opportunity to go again. Open late!

Jeremy Saal, Rating: 5/5, Visited: a year ago
Experience: These were the best gyros we ever had. In fact, we dined there twice since the food was so good. This place was recommended to us by a local chef, who mentioned their pitas are made the traditional way, rather than mass-produced. The quality of the food and flavor were superb. Next time we're in Athens, this will be our first dining spot.

Allan Venegas, Rating: 5/5, Visited: 6 months ago
Experience: We came here during a food and wine tour we had booked and now I know why. This gyro is the best we have ever had anywhere. They show how these pitas are baked - fluffy yet crispy - packed with different meats. We only came here for a quick bite, but we definitely are going back before we leave Athens again. Even Gordon Ramsey came here in the past! Highly recommend

Sila A., Rating: 5/5, Visited: a year ago
Experience: Delicious gyros with quality ingredients. We tried the chicken pita and pork special, both were very tasty. The pitas are a bit small, consider ordering 2 per person if you are hungry.

Vlad M., Rating: 4/5, Visited: 7 months ago
Experience: Pita and sauce on the pita is nice.different.
The fries and salad is okay. But the cheese is not fluffy enough.
Overall it is very cheap!


Based on the above most relevant comments and overall data, write a small paragraph stating the reasons if and why you want to visit that restaurant.
You are an abrasive software engineer named Takis and you do not like to spend time or money in mediocre restaurants, and you value VFM above all.
Large portions get an extra credit, expensive prices are not. Recent reviews count more.
Write in a software engineering style making jokes and puns, while also referencing the reviewers.

Some example code

In order to build the above, which was the easy part (devops here, not front end dev), I used node js, express and the direct library from OpenAI. I tried other implementations, like Chatgpt API, but at the time I wrote this app, I opted from ES6 and not Typescript (again devops here), which didn’t actually work for me.

 try {
    logger.info('ENABLECHATGPT', process.env.ENABLE_CHATGPT)
    let prompt = `Restaurant: '${data.name}' near ${data.vicinity}, Overall Rating: ${data.rating} from ${data.totalReviews} Reviews, types: ${data.type}\n`
    data.comments.map(comment => {
      prompt += `\n${comment.author_name}, Rating: ${comment.rating}/5, Visited: ${comment.relative_time_description} \nExperience: ${comment.text}\n`
    });
    prompt += `\n\n${template}`
    logger.info('prompt', prompt)
    let completion = { data: { usage: { prompt_tokens: 498, completion_tokens: 116, total_tokens: 614 }, choices: [{ text: "\n\nDefault Text!" }] } }
    if (process.env.ENABLE_CHATGPT == 'true') {
      logger.info('Sending to ChatGPT')
      completion = await openai.createCompletion({
        model: "text-davinci-003",
        prompt: prompt,
        max_tokens: 1000,
        temperature: 0.6,
        top_p: 1,
        n: 1,
        stream: false,
        logprobs: null,
        stop: null
      });
    }
    return completion.data.choices[0].text
  } catch (e) {
    logger.error(e)
    return "Mr. Takis is experiencing some electric intestinal problems right now and he's stuck at the electrical discharge seat..."
  }

Implementation

I chose all JS for this. For various reasons. One reason is that I already had a server with node js installed. Another is that I had worked on a previous project using node js and I had a template ready (and I do not develop stuff that often any more since I’ve officially become a devops for a long time now). And lastly because I’m certain people use JS more often than they are admitting.

I’ll update how I did the implementation soon, in the meantime, have a look at the app and give me a shout if you like it. I’ll just stop here with the following:

ChatGPT is good at “helping” write SIMPLE code.

That is because, I hadn’t worked with ReactJS before that extensively, let alone use some modules I haven’t seen before. So, I did the obvious thing, and asked ChatGPT to write me an “google maps autocomplete react js app widget”.

The above didn’t work out so well. Since the trained model for gpt when I was asking it was from 2020, that’s ages in the js, let alone react world. Therefore, it was hallucinating. And here’s when the frustration starts to kick in… The prompt I gave it to start was the following:

create a sample react app with a form and autocomplete from googles places api only for restaurants

And the response was this:

npx create-react-app restaurant-finder
cd restaurant-finder
npm install @google/maps react-google-autocomplete

And the code:

import React from 'react';
import { GoogleMap, LoadScript, Autocomplete } from '@react-google-maps/api';

const RestaurantFinder = () => {
  const onLoad = (autocomplete) => {
    console.log('autocomplete: ', autocomplete);
  };

  return (
    <LoadScript googleMapsApiKey="YOUR_API_KEY">
      <GoogleMap center={{ lat: 37.7749, lng: -122.4194 }} zoom={12}>
        <Autocomplete
          onLoad={onLoad}
          onPlaceChanged={() => {
            console.log('place changed');
          }}
          options={{
            types: ['establishment'],
            componentRestrictions: { country: 'us' },
          }}
        />
      </GoogleMap>
    </LoadScript>
  );
};

export default RestaurantFinder;

For the App.js:

import React from 'react';
import RestaurantFinder from './RestaurantFinder';

function App() {
  return (
    <div className="App">
      <RestaurantFinder />
    </div>
  );
}

export default App;

and here’s where things start to get weird. GPT searched for google maps, autocomplete and react and it autogenerated something that didn’t quite help. A simple autocomplete form which was conflicting locally with the current react installation.

After a lot of trial and error, and frustrated error messages, I tried the following:

create a sample react app using googlemaps/react-wrapper and not google-maps-react, with a form and autocomplete from googles places api only for restaurants.

That is when GPT went fully hallucinating. It created a react app, that was using the googlemaps/react-wrapper but only by approximation, as it was trying to extend the google-maps-react functionality. Obviously this did not work at all. The previous attempts, weren’t working either, as the react app had moved ahead a lot, but, even if you do try to “guess” the correct combo of packages it was referring to, again, for me it did not work.

Must Feed