Building a blog in Go Part 4

Extra Credit

I did not want to leave with you hanging with no unit tests or FrontMatter parsing. Let's do some very basic testing on some of the more important functions.

In the root of our project, create a new go file named main_test.go.

1touch main_test.go

I am not going to provide a ton of guidance on testing with go. If you want to learn more visit Learn Go With Tests. I will however provide you some basic tests. In your new file let's add some code.

 1func TestReadMarkDown(t *testing.T) {
 2  // create a fake file system with in memory map
 3 fakeFS := fstest.MapFS{
 4  "hello-world1.md": {Data: []byte("# hi")},
 5  "hello-world2.md": {Data: []byte("# hola")},
 6 }
 7 
 8  // pass in the fake filesystem an let's see what we got back
 9 gotMap, err := readMarkdown(fakeFS)
10 if err != nil {
11  t.Errorf("error reading markdown from test %v", err)
12 }
13 
14  // lets define what we should get
15 wantMap := map[string][]byte{
16  "hello-world1": []byte("# hi"),
17  "hello-world2": []byte("# hola"),
18 }
19 
20  // now compare what we should get vs what we got
21 for i, wantval := range wantMap {
22  gotval, found := gotMap[i]
23  if !found {
24   t.Errorf("error did not find %s", i)
25  }
26
27  if !bytes.Equal(wantval, gotval) {
28   t.Errorf("byte values not equal %s & %s", string(wantval), string(gotval))
29  }
30 }
31}

In this code, we are using some great features in Go. One of them is to the fstest library. This library provides all sorts of code to allow you to test your code easier. Part of the reason we passed in the fs.FS interface into readMarkdown() was to make this function easier to test. Let's give this a go, in the terminal run

1go test -v

The test should pass.

Let's refactor this map checking into its own standalone function. Go ahead and create a function called equalByteMaps(a,b map[string]byte) bool . The goal here will be to:

1. Check the length of each map, if they are not the same return false
2. Iterate through each value in a, check to see if that value exists in b
3. Compare the two values byte using bytes.equal

 1func equalByteMaps(a, b map[string][]byte) bool {
 2 // compare lengths, ensure the same
 3 if len(a) != len(b) {
 4  return false
 5 }
 6
 7 // iterate through each value in a
 8 for akey, aval := range a {
 9  bval, found := b[akey]
10  if !found {
11   return false
12  }
13
14  if !bytes.Equal(aval, bval) {
15   return false
16  }
17 }
18 return true
19}

Now let's refactor our original test.

 1func TestReadMarkDown(t *testing.T) {
 2 fakeFS := fstest.MapFS{
 3  "hello-world1.md": {Data: []byte("# hi")},
 4  "hello-world2.md": {Data: []byte("# hola")},
 5 }
 6
 7 gotMap, err := readMarkdown(fakeFS)
 8 if err != nil {
 9  t.Errorf("error reading markdown from test %v", err)
10 }
11
12 wantMap := map[string][]byte{
13  "hello-world1": []byte("# hi"),
14  "hello-world2": []byte("# hola"),
15 }
16
17 if !equalByteMaps(wantMap, gotMap) {
18  t.Errorf("failed map comparsion")
19 }
20}

That helped clean things up a bit and has provided us with a helper function that we can reuse in the future.

Let's also tackle our handlers... If you made it this far, we have made some mistakes for the sake of simplicity. The handlers we have written are simply running out of the main function. How do you feel about functions being called out of main? The answer here is to clean up our app a little bit and make it more testable.

Let's move our handlers out of main function. Go ahead and create two new files in the root of the project, routes.go, handlers.go, & helpers.go.

1touch routes.go handlers.go helpers.go

Let's start with our helpers.go. With this file, I wanted to take a moment to clean up the overall project and have the main() function be slimmer. Let's move both readMarkdown() and slugToTitle() functions from main.go to helpers.go. If that sounded confusing, just make sure helpers.go looks like this:

 1package main
 2
 3import (
 4 "io/fs"
 5 "strings"
 6
 7 "golang.org/x/text/cases"
 8 "golang.org/x/text/language"
 9)
10
11func readMarkdown(fSys fs.FS) (map[string][]byte, error) {
12 cache := make(map[string][]byte)
13 files, err := fs.Glob(fSys, "*.md") // read all .md files from the file system
14 if err != nil {
15  return nil, err
16 }
17
18 for _, f := range files {
19  data, err := fs.ReadFile(fSys, f)
20  if err != nil {
21   return nil, err
22  }
23
24  // cache["firstpost"] = []byte("...markdown content...")
25  cache[strings.TrimSuffix(f, ".md")] = data
26 }
27
28 return cache, nil
29}
30
31func slugToTitle(slug string) string {
32 slug = strings.TrimSuffix(slug, ".md")
33 slug = strings.ReplaceAll(slug, "-", " ")
34 caser := cases.Title(language.AmericanEnglish)
35 return caser.String(slug)
36}

Next up, let introduce a new struct in our project called application. This will contain references to our cache and other dependencies and allow us to both access them and test them. At the top of main.go add the following just above the config structure.

1// stores a ref to our template sets
2type application struct {
3 templateCache map[string]*template.Template
4}
5// no change here, just noted for ref
6type config struct {
7 port int
8 path string
9}

We are going to make our handlers (which are just functions) be methods on the application struct.

Next go to our handlers.go file and add the following code.

 1package main
 2
 3import (
 4 "net/http"
 5)
 6
 7func (a *application) homeHandler(w http.ResponseWriter, r *http.Request) {
 8 w.Write([]byte("hi"))
 9}
10
11func (a *application) postHandler(w http.ResponseWriter, r *http.Request) {
12 w.Write([]byte("hi"))
13}

This func (a *application) name is how we implement a methond on a struct. By doing this, we are able to access everything that application has access to. Remember the cache we added earlier? We can now access that directly inside our handlers.

Let's modify these handlers to make them a little more useful.

 1func (a *application) homeHandler(w http.ResponseWriter, r *http.Request) {
 2 // fetch our template set from our cache
 3 ts := a.templateCache["home"]
 4
 5 // create a homepage data struct and assign a title
 6 pageData := HomePageData{
 7  Title: "Home",
 8 }
 9
10 // grab our post titles from our markdown cache
11 for k := range a.markdownCache {
12  pageData.Urls = append(pageData.Urls, k)
13 }
14
15 // execute our template with the page data
16 err := ts.ExecuteTemplate(w, "base", pageData)
17 if err != nil {
18  http.Error(w, err.Error(), http.StatusInternalServerError)
19 }
20}

None of this should look unfamiliar, this was pretty much already in the main function. Next, let's tackle the post handler as that was a little more complicated.

 1func (a *application) postHandler(w http.ResponseWriter, r *http.Request) {
 2 slug := r.PathValue("slug")
 3 if slug == "" {
 4  http.Error(w, "Post Not Found", http.StatusNotFound)
 5  return
 6 }
 7
 8 // fetch our template set
 9 ts := a.templateCache["posts"]
10
11 // fetch our markdown data
12 data, found := a.markdownCache[slug]
13 if !found {
14  http.Error(w, "Post Not Found", http.StatusNotFound)
15  return
16 }
17
18 var buf bytes.Buffer
19 if err := goldmark.Convert(data, &buf); err != nil {
20  log.Printf("error converting markdown for %q: %v", slug, err)
21  http.Error(w, "Something went wrong rendering this post", http.StatusInternalServerError)
22  return
23 }
24
25 p := PostPageData{
26  Title:   slugToTitle(slug),
27  Content: template.HTML(buf.String()),
28 }
29
30 // set the content type
31 w.Header().Set("Content-Type", "text/html")
32 if err := ts.ExecuteTemplate(w, "base", p); err != nil {
33  fmt.Printf("Error rendering template %v\n", err)
34  return
35 }
36}

There was only a few modifications. This was for the most part copy and paste (minus a few tweaks). We have our handlers now!

Time to tackle our routes.go file that handles our routing to the handlers. Go ahead and add the following code to our routes file.

1func (a *application) routes() *http.ServeMux {
2 mux := http.NewServeMux()
3 
4 mux.HandleFunc("/", a.homeHandler)
5 mux.HandleFunc("/posts/{slug}", a.postHandler)
6
7 return mux
8}

Again, we are building a method off the application struct called routes that returns a *http.ServeMux . What the hell is a ServeMux you ask?

In Go, ServeMux is an HTTP request multiplexer that acts as a router, directing incoming requests to the correct handler based on the URL. It matches a request's URL against a list of registered patterns and then dispatches the request to the handler that matches the most closely

It is a router, plain and simple. When we get a request to "/" we route it to our homeHandler function.

With all of that out of the way, we need to make some rather large changes to the main.go file to support all of this. Go ahead and pop open your main.go.

Make the following changes (if you get lost I will paste the entire main.go below)

1. Remove the handlers from our main() function of main.go file.

2. At the top of our main() create a variable for our application struct var app application

3. After the call to create a markdownCache go ahead and assign the app.markdownCache = markdownCache

   1// read our markdown files
   2 markdownCache, err := readMarkdown(os.DirFS(cfg.path))
   3 if err != nil {
   4  log.Fatalf("failed to read markdown files: %v", err)
   5  return
   6 }
   7
   8 // add our markdown cache to our app
   9 app.markdownCache = markdownCache
   

4. Get rid of these lines of code

    1 // create our homePageDate struct
    2 homePageData := HomePageData{
    3  Title: "Home",
    4 }
    5
    6 // loop through our cache and add each url to the homePageData struct
    7 for k := range markdownCache {
    8  homePageData.Urls = append(homePageData.Urls, k)
    9 }
   10
   11 // sort our urls
   12 sort.Strings(homePageData.Urls)
   

   Lastly towards the bottom of main() make these changes

    1app.templateCache["home"] = homeTmpl
    2 app.templateCache["posts"] = postTmpl
    3
    4 // create a new instance of http.server
    5 srv := &http.Server{
    6  Addr:    fmt.Sprintf(":%d", cfg.port),
    7  Handler: app.routes(),
    8 }
    9
   10 // start our server on our port or default to 5040
   11 fmt.Printf("starting server on port :%d\n", cfg.port)
   12
   13 // use our servemux and start
   14 if err := srv.ListenAndServe(); err != nil {
   15  fmt.Printf("Server crashed with error %v\n", err)
   16 }
   

That was alot of change but it drastically cut down some of code in main.go and main(). If you are still lost regarding handlers and servemuxes, a fantastic article on handlers and servemux is this one Alex Edwards Introduction to handlers and servemuxes in go

Here is our complete main.go file.

 1package main
 2
 3import (
 4 "flag"
 5 "fmt"
 6 "html/template"
 7 "log"
 8 "net/http"
 9 "os"
10)
11
12type application struct {
13 templateCache map[string]*template.Template
14 markdownCache map[string][]byte
15}
16
17type config struct {
18 port int
19 path string
20}
21
22type HomePageData struct {
23 Title string
24 Urls  []string
25}
26
27type PostPageData struct {
28 Title   string
29 Content template.HTML
30}
31
32func main() {
33 var cfg config
34 var app application
35
36 flag.IntVar(&cfg.port, "port", 5040, "port to listen on")
37 flag.StringVar(&cfg.path, "content-path", "./content", "path to markdown content")
38 flag.Parse()
39
40 // read our markdown files
41 markdownCache, err := readMarkdown(os.DirFS(cfg.path))
42 if err != nil {
43  log.Fatalf("failed to read markdown files: %v", err)
44  return
45 }
46
47 // add our markdown cache to our app
48 app.markdownCache = markdownCache
49
50 // create a home template set by parsing both base and home
51 homeTmpl, err := template.ParseFiles(
52  "templates/base.tmpl",
53  "templates/home.tmpl",
54 )
55 if err != nil {
56  log.Fatalf("failed to parse home templates: %v", err)
57 }
58
59 // create a post template set by parsing both base and post
60 postTmpl, err := template.ParseFiles(
61  "templates/base.tmpl",
62  "templates/post.tmpl",
63 )
64 if err != nil {
65  log.Fatalf("failed to parse post templates: %v", err)
66 }
67
68 app.templateCache["home"] = homeTmpl
69 app.templateCache["posts"] = postTmpl
70
71 srv := &http.Server{
72  Addr:    fmt.Sprintf(":%d", cfg.port),
73  Handler: app.routes(),
74 }
75
76 // start our server on our port or default to 5040
77 fmt.Printf("starting server on port :%d\n", cfg.port)
78 // use the default serve mux to start a server on port 5040 and check for errors
79 if err := srv.ListenAndServe(); err != nil {
80  fmt.Printf("Server crashed with error %v\n", err)
81 }
82}

The work here we have done to refactor the app will help a bit when it comes to testing and just general orginization. Let's go ahead and try our tests and start our server to make sure everything works as intended.

1go test -v ./...
2go run .

While my test worked fine, running the app generated a panic: panic: assignment to entry in nil map this is a pretty descriptive error. Let's take a look at the code, we are assigning template sets to the map before initializing it. Go ahead and add these lines

1app.templateCache = make(map[string]*template.Template) // create a new map
2//existing code
3app.templateCache["home"] = homeTmpl
4app.templateCache["posts"] = postTmpl

After making this change our server should start up with no issues. If you are using air you should be in business. In the terminal we can issue this command to ensure everything is rendering correctly.

1curl http://localhost:5040

This was my (part) of the output. The thing we want to see here is that we are rendering our template and that the data is rendering (our list of links).

 1<body>
 2  <div class="container"> 
 3  <h1>Hi 👋</h1>
 4  <p>Welcome to our blog! Here you will find a collection of posts for all things interesting. I hope you enjoy!</p>    
 5      <ul>
 6        <li><a href="/posts/hotdogs">hotdogs</a></li>
 7      </ul>  
 8      <ul>
 9        <li><a href="/posts/icecream">icecream</a></li>
10      </ul>  
11      <ul>
12        <li><a href="/posts/tacos">tacos</a></li>
13      </ul>
14  </div>

We did a ton of refactoring work, but what about testing our handlers? We still need to do that. In our main_test.go let's add a test to ensure that our tests can hit our home handler

 1func TestHomeHandler(t *testing.T) {
 2  // define a template named "base" and parse this content
 3 homeTmpl := template.Must(template.New("base").Parse(`
 4 {{define "base"}}<html><body>{{range .Urls}}<p>{{.}}</p>{{end}}</body></html>{{end}}
 5 `))
 6 
 7  // create an app struct and define our caches
 8 app := application{
 9  templateCache: map[string]*template.Template{"home": homeTmpl},
10  markdownCache: map[string][]byte{"hello-world": {}, "second-post": {}},
11 }
12 
13  // create a request and response
14 req := httptest.NewRequest(http.MethodGet, "/", nil)
15 rr := httptest.NewRecorder()
16 app.homeHandler(rr, req)
17 
18  // ensure we get statusok
19 if rr.Code != http.StatusOK {
20  t.Fatalf("status = %d, want %d", rr.Code, http.StatusOK)
21 }
22 
23  // check for hello-world rendering
24 body := rr.Body.String()
25 if !strings.Contains(body, "hello-world") || !strings.Contains(body, "second-post") {
26  t.Fatalf("body missing urls, got %q", body)
27 }
28}

We have a few new concepts here.

1. We are leveraging more new features related to testing by using the httptest package. This package provides us the capability to create a new request, record that request and then check the output of that request to ensure it is what we expect.
2. Ideally our tests are light weight and we do not introduce a ton of their own dependencies such as calling other functions from our code base.

Let's add the few last couple of tests

 1func TestPostHandler_OK(t *testing.T) {
 2 postTmpl := template.Must(template.New("base").Parse(`{{define "base"}}<h1>{{.Title}}</h1>{{.Content}}{{end}}`))
 3 app := application{
 4  templateCache: map[string]*template.Template{"posts": postTmpl},
 5  markdownCache: map[string][]byte{"hello-world": []byte("# title\n\ncontent")},
 6 }
 7
 8 req := httptest.NewRequest(http.MethodGet, "/posts/hello-world", nil)
 9 req.SetPathValue("slug", "hello-world")
10 rr := httptest.NewRecorder()
11 app.postHandler(rr, req)
12
13 if rr.Code != http.StatusOK {
14  t.Fatalf("status = %d, want %d", rr.Code, http.StatusOK)
15 }
16 if ct := rr.Header().Get("Content-Type"); ct != "text/html" {
17  t.Fatalf("Content-Type = %q, want text/html", ct)
18 }
19 if !strings.Contains(rr.Body.String(), "<h1>Hello World</h1>") {
20  t.Fatalf("rendered body unexpected: %q", rr.Body.String())
21 }
22}

and lastly for not found posts.

 1func TestPostHandler_NotFound(t *testing.T) {
 2 app := application{templateCache: map[string]*template.Template{"posts": template.Must(template.New("base").Parse(`{{define "base"}}ok{{end}}`))}}
 3
 4 req := httptest.NewRequest(http.MethodGet, "/posts/missing", nil)
 5 req.SetPathValue("slug", "missing")
 6 rr := httptest.NewRecorder()
 7 app.postHandler(rr, req)
 8
 9 if rr.Code != http.StatusNotFound {
10  t.Fatalf("status = %d, want %d", rr.Code, http.StatusNotFound)
11 }
12}

Run these tests and ensure they pass.

1go test -v

Wrapping up

In this extra credit we actually did quite a bit, we refactored the application to be a bit more testable and we built out some tests to ensure any changes in the future get caught. Hopefully this gives you a good base to build on in the future. I hope you enjoyed and learned something!