Flex and Bison

Purpose of Flex and Bison

Flex and Bison allow for the generation of programs that process structured data. A prime example for such programs are interpreters or compilers of programming languages. Other applications are possible such as a loader for specific file formats.

Flex is a successor of lex. They are both lexer generators.

Bison is a successor of Yacc. They are both parser generators.

A lexer reads a string of characters and given rules, returns token matched from the character stream. Token are groups of characters. The rules applied to convert characters into token are defined by the programmer. The file ending .l is usually used for flex files.

A parser takes a stream of token and given rules, recognized constructs in the language it is supposed to parse. A parser could for example recognize function definitions or definitions of if-statements. Parts or rules or entire rules can be bound to actions. As soon as a rule is recognized, the action is executed. Actions could be code generation, for example assembler code could be output when a rule was parsed.

Usage of Flex and Bison

When compiling, the first step is to use the bison parser generator on the .y file. This will generate a .c and a .h file.

The second step is to use the flex lexer generator on the .l file. The .l file has to import the .h file that was generated by the bison run in the first step. The reason for importing that generated header is that the header defines constants for token that have to be returned by the generated lexer.

This means that first the grammar rules are processed and then after that the lexer rules are processed. This is kind of backwards. One would expect that first the token are defined and then the rules are build on top of the available token. The process is more or less backwards. You defined the rules using token and at that point you do not care how the token are defined. In the second step, when you know which token the grammar rules need, you define what the token look like by designing appropriate lexer rules.


On MacOS you have to link agains -ll instead of -lfl

On MacOS, your parser file has to define yyerror() and yylex()

   #include <stdio.h>
   void yyerror(const char* msg) {
   	  printf("bla %s\n", msg);
      fprintf(stderr, "bli %s\n", msg);
   int yylex();


HTTP HTTPS GET Requests with Poco

The standard Poco GET example has one major deficit, I can only succesfully execute a GET request against a server that supports HTTP. Most servers do not provide their content using HTTP any more. Instead a HTTPS GET request has to be executed.

In order to adjust the Poco HTTP example to use HTTPS, follow the steps outlined in https://stackoverflow.com/questions/10875938/how-to-use-openssl-in-poco-c-library-correctly

The example below is not good code at all whatsoever but it shows how to use the HTTPSClientSession instead of the HTTPClientSession to leverage the HTTPS protocol instead of the HTTP protocol.

#include <iostream>

#include "Poco/MD5Engine.h"
#include "Poco/DigestStream.h"

#include "Poco/Net/HTTPClientSession.h"
#include "Poco/Net/HTTPSClientSession.h"
#include "Poco/Net/HTTPMessage.h"
#include "Poco/Net/HTTPRequest.h"
#include "Poco/Net/HTTPResponse.h"
#include "Poco/StreamCopier.h"
#include "Poco/Path.h"
#include "Poco/URI.h"
#include <Poco/Exception.h>

using namespace std;

using Poco::Net::HTTPClientSession;
using Poco::Net::HTTPSClientSession;
using Poco::Net::HTTPRequest;
using Poco::Net::HTTPResponse;
using Poco::Net::HTTPMessage;
using Poco::StreamCopier;
using Poco::Path;
using Poco::URI;
using Poco::Exception;

int main(int argc, char** argv) {

  try {

    std::cout << "Hello world2" << std::endl;

    //URI uri("https://www.google.de:80/index.html");
    //URI uri("https://www.play-hookey.com:80/htmltest/");
    //URI uri("http://www.brainjar.com:80/java/host/test.html");


    //URI uri("http://us.imdb.com:80/Title?Toy%20Story%20(1995)");
    //URI uri("https://www.imdb.com:443/Title?Toy%20Story%20(1995)");
    //URI uri("https://www.imdb.com/find?q=Toy%20Story%20(1995)");
    URI uri("https://www.imdb.com/find?q=Se7en%20(1995)");

    //URI uri("https://www.imdb.com:443/find?q=Toy%20Story%20(1995)");
    //URI uri("https://www.imdb.com:443/find");
    //URI uri("");
    //URI uri("https://www.imdb.com:443");
    //URI uri("https://www.imdb.com");
    //URI uri("www.imdb.com");

    //URI uri("https://stackoverflow.com/");

    //URI uri("https://github.com/");

    // prepare path
    string path(uri.getPathAndQuery());
    if (path.empty())
      path = "/";

    std::cout << "host " << uri.getHost() << " port " << uri.getPort()
        << " path " << path << std::endl;

    HTTPRequest request(HTTPRequest::HTTP_GET, path, HTTPMessage::HTTP_1_1);

    // HTTP
    //HTTPClientSession session(uri.getHost(), uri.getPort());

    // HTTPS
    const Poco::Net::Context::Ptr context = new Poco::Net::Context(
        Poco::Net::Context::CLIENT_USE, "", "", "",
        Poco::Net::Context::VERIFY_NONE, 9, false,
    HTTPSClientSession session(uri.getHost(), uri.getPort(), context);

    std::cout << "session.sendRequest" << std::endl;

    std::cout << "session.receiveResponse" << std::endl;

    HTTPResponse response;
    std::istream& rs = session.receiveResponse(response);

    std::cout << "session.getStatus" << std::endl;
    std::cout << response.getStatus() << " " << response.getReason()
        << std::endl;

    //StreamCopier::copyStream(rs, std::cout);

    std::string responseAsString(std::istreambuf_iterator<char>(rs), { });

  } catch (Exception &ex) {
    std::cerr << ex.displayText() << std::endl;
    return -1;

  return 0;


Conan Package Manager for C++

https://docs.conan.io/en/latest/getting_started.html contains a good introduction to Conan.

Installing Conen on MacOS can be done via brew:

brew update
brew install conan

Conan is controlled by a file called conanfile.txt. It is comparable to a maven pom or the package.json file for the node package manager.

The conanfile.txt from the tutorial that works with CMake is duplicated below



You can now execute conan install in the folder that contains the conanfile.txt to exceute Conan. It will install all dependencies and call all generators listed in the conanfile.txt.

If Conan fails to download precompiled binaries, sometimes it is possible to tell conan to build the depencencies from code:

conan install Poco/1.9.0@pocoproject/stable --build missing

To use Conan in combination with CMake, the conanfile.txt has to create a conanbuildinfo.cmake file using a generator for CMake. That file is then used from within CMake’s CMakeLists.txt file. When CMake builds the project, it is able to call Conan.

A CMakeLists.txt file that imports the generated conanbuildinfo.cmake is given below:

cmake_minimum_required (VERSION 2.6)

project (PocoTest)



add_executable(PocoTest PocoTest.cpp)
target_link_libraries(PocoTest ${CONAN_LIBS})

I think the workflow now is, whenever you need a new library in your project, add the dependency to Conans conanfile.txt. Then call conan install so Conan can download and install the new dependency and also generates an updated conanbuildinfo.cmake for CMake. Then build your project using CMake to include the newly provided dependencies.


Have a CMakeLists.txt file in each directory that
contains project source code. The root folder of all the
directories containing CMakeLists.txt files is referred
to as the source directory.

The source directory along with the binary directory
constitute the set of folders that CMake performs builds
in. A out-of-source build is one where CMake is reading
from the source directory and writing artefacts to the
binary directory. This is the default. By default CMake
will not write into the source directory, only read from
it. A in-source build is where CMake is configured to
write artifacts into the source directory, in other words
where source and binary directory are the same.

out-of-source builds are easier to maintain, because by
checking in the source directory to a version control
system and leaving the binary directory unversioned,
you are sure to not commit artifacts. Also you can
erase the binary directory and clean your project that

CMakeLists.txt files contain one or more CMake commands

A command has the syntax: command (args…) where
command is the name of a command and args is a white-
space separated list of arguments. (Arguments with
embedded white-space should be double quoted).
Commands are also used for controlling the program flow
if() else() endif()

Variables are defined using the ${VAR} syntax.
Assigning a value is done using set(Foo a b c) which
sets the variable Foo to the list of values a b and c.
Using a variable: command(${Foo}) which is equivalent to
command(a b c) if Foo has the values a b c set.

Environment variables can be used: $ENV{VAR}

Building from the command line:
CMake offers the –build option, which is described as a
convenience that allows you to build your project from the
command line even if this requires launching an IDE.
The syntax is:
cmake –build <dir> [options] [– [native-options]]
cmake –build . — -v
will build in the current working directory and pass the
parameter -v to the underlying build tool.

Open Questions

Can I build with CMake?

Q: If CMake is a tool to generate build configurations
for different platforms, why can eclipse use cmake to
directly build a binary for my project?
A: CMake offers the –build option, which is described as a
convenience that allows you to build your project from the
command line even if this requires launching an IDE

Where does the build folder and the contained folders
in my EclipseCDT project come from?

Q: When looking at my CMake project in EclipseCDT, it
has a build folder and a lot of folders and files below
it, where do these folders come from?
A: ???

Eclipse CDT C++ CMake on Mac

Using Eclipse CDT on mac along with the g++ compiler is a very good option if you know your Eclipse hotkeys from programming in Java for example or if you are looking for an alternative to XCode.

Eclipse CDT allows you to create CMake projects. This post sums up the traps I got caught up in before getting everything to work in the hopes it will help others to not make the errors I made.

On your Mac, first install cmake and ninja

brew update
brew install cmake
brew install cask cmake
brew install ninja

If the brew commands fail, try repeating them, this sometimes fixes install issues.

Next, start Eclipse CDT from the console instead of from the finder (CMD + space). To start Eclipse CDT from a console, execute the command

open -a EclipseCDT.app

Opening Eclipse CDT from the finder will not add /usr/local/bin to Eclipse’s Path! Opening Eclipse CDT from the command line will add /usr/local/bin to Eclipse’s Path. /usr/local/bin has to be part of the PATH as only then can Eclipse CDT execute ninja and cmake. This is summed up in the post: https://bugs.eclipse.org/bugs/show_bug.cgi?id=532419

You are now able to create a new C/C++ CMake project from within eclipse, clean it, build it and run the executable.

C++ on MacOS with Eclipse CDT

The Eclipse C++ Developer Tooling adds C++ support to Eclipse.

The installation URL is https://download.eclipse.org/tools/cdt/releases/9.6/ using this URL, you can install CDT form within Eclipse using the “Install new Software” feature.

Eclipse CDT requires a C++ compiler being installed on the System. Under MacOS the easiest way to install a compiler is to install XCode from the app store. This also installs the lldb debugger which Eclipse CDT has support for. The latest version of XCode does not come with the gdb debugger which is the default debugger in a vanilla installation of CDT. That means you have to change Eclipse CDT to use the lldb debugger instead of gdb.

In order to debug using lldb, you have to edit the workspace settings and change the default debugger from gdb to lldb. The settings for the debugger are contained in Eclipse Menu > Preferences > Run/Debug > Launching > Default Launchers. In the submenu, change the Debug option to lldb and save.