blob: 888c341701da49f7c5f2d953144c4ba675ffe3a4 [file] [log] [blame]
Brendan Higginsc23a2832019-09-23 02:02:45 -07001.. SPDX-License-Identifier: GPL-2.0
2
3===========
4Using KUnit
5===========
6
7The purpose of this document is to describe what KUnit is, how it works, how it
8is intended to be used, and all the concepts and terminology that are needed to
9understand it. This guide assumes a working knowledge of the Linux kernel and
10some basic knowledge of testing.
11
12For a high level introduction to KUnit, including setting up KUnit for your
13project, see :doc:`start`.
14
15Organization of this document
16=============================
17
Daniel Latypov1f0e9432020-11-23 14:57:59 -080018This document is organized into two main sections: Testing and Common Patterns.
19The first covers what unit tests are and how to use KUnit to write them. The
20second covers common testing patterns, e.g. how to isolate code and make it
21possible to unit test code that was otherwise un-unit-testable.
Brendan Higginsc23a2832019-09-23 02:02:45 -070022
23Testing
24=======
25
26What is KUnit?
27--------------
28
29"K" is short for "kernel" so "KUnit" is the "(Linux) Kernel Unit Testing
30Framework." KUnit is intended first and foremost for writing unit tests; it is
31general enough that it can be used to write integration tests; however, this is
32a secondary goal. KUnit has no ambition of being the only testing framework for
33the kernel; for example, it does not intend to be an end-to-end testing
34framework.
35
36What is Unit Testing?
37---------------------
38
39A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is a test that
40tests code at the smallest possible scope, a *unit* of code. In the C
41programming language that's a function.
42
43Unit tests should be written for all the publicly exposed functions in a
44compilation unit; so that is all the functions that are exported in either a
45*class* (defined below) or all functions which are **not** static.
46
47Writing Tests
48-------------
49
50Test Cases
51~~~~~~~~~~
52
53The fundamental unit in KUnit is the test case. A test case is a function with
54the signature ``void (*)(struct kunit *test)``. It calls a function to be tested
55and then sets *expectations* for what should happen. For example:
56
57.. code-block:: c
58
59 void example_test_success(struct kunit *test)
60 {
61 }
62
63 void example_test_failure(struct kunit *test)
64 {
65 KUNIT_FAIL(test, "This test never passes.");
66 }
67
68In the above example ``example_test_success`` always passes because it does
69nothing; no expectations are set, so all expectations pass. On the other hand
70``example_test_failure`` always fails because it calls ``KUNIT_FAIL``, which is
71a special expectation that logs a message and causes the test case to fail.
72
73Expectations
74~~~~~~~~~~~~
75An *expectation* is a way to specify that you expect a piece of code to do
76something in a test. An expectation is called like a function. A test is made
77by setting expectations about the behavior of a piece of code under test; when
78one or more of the expectations fail, the test case fails and information about
79the failure is logged. For example:
80
81.. code-block:: c
82
83 void add_test_basic(struct kunit *test)
84 {
85 KUNIT_EXPECT_EQ(test, 1, add(1, 0));
86 KUNIT_EXPECT_EQ(test, 2, add(1, 1));
87 }
88
89In the above example ``add_test_basic`` makes a number of assertions about the
90behavior of a function called ``add``; the first parameter is always of type
91``struct kunit *``, which contains information about the current test context;
92the second parameter, in this case, is what the value is expected to be; the
93last value is what the value actually is. If ``add`` passes all of these
94expectations, the test case, ``add_test_basic`` will pass; if any one of these
Randy Dunlap873ddeb2020-10-28 10:43:19 -070095expectations fails, the test case will fail.
Brendan Higginsc23a2832019-09-23 02:02:45 -070096
97It is important to understand that a test case *fails* when any expectation is
98violated; however, the test will continue running, potentially trying other
99expectations until the test case ends or is otherwise terminated. This is as
100opposed to *assertions* which are discussed later.
101
102To learn about more expectations supported by KUnit, see :doc:`api/test`.
103
104.. note::
105 A single test case should be pretty short, pretty easy to understand,
106 focused on a single behavior.
107
108For example, if we wanted to properly test the add function above, we would
109create additional tests cases which would each test a different property that an
110add function should have like this:
111
112.. code-block:: c
113
114 void add_test_basic(struct kunit *test)
115 {
116 KUNIT_EXPECT_EQ(test, 1, add(1, 0));
117 KUNIT_EXPECT_EQ(test, 2, add(1, 1));
118 }
119
120 void add_test_negative(struct kunit *test)
121 {
122 KUNIT_EXPECT_EQ(test, 0, add(-1, 1));
123 }
124
125 void add_test_max(struct kunit *test)
126 {
127 KUNIT_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX));
128 KUNIT_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN));
129 }
130
131 void add_test_overflow(struct kunit *test)
132 {
133 KUNIT_EXPECT_EQ(test, INT_MIN, add(INT_MAX, 1));
134 }
135
136Notice how it is immediately obvious what all the properties that we are testing
137for are.
138
139Assertions
140~~~~~~~~~~
141
142KUnit also has the concept of an *assertion*. An assertion is just like an
143expectation except the assertion immediately terminates the test case if it is
144not satisfied.
145
146For example:
147
148.. code-block:: c
149
150 static void mock_test_do_expect_default_return(struct kunit *test)
151 {
152 struct mock_test_context *ctx = test->priv;
153 struct mock *mock = ctx->mock;
154 int param0 = 5, param1 = -5;
155 const char *two_param_types[] = {"int", "int"};
156 const void *two_params[] = {&param0, &param1};
157 const void *ret;
158
159 ret = mock->do_expect(mock,
160 "test_printk", test_printk,
161 two_param_types, two_params,
162 ARRAY_SIZE(two_params));
163 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ret);
164 KUNIT_EXPECT_EQ(test, -4, *((int *) ret));
165 }
166
167In this example, the method under test should return a pointer to a value, so
168if the pointer returned by the method is null or an errno, we don't want to
169bother continuing the test since the following expectation could crash the test
170case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case if
171the appropriate conditions have not been satisfied to complete the test.
172
173Test Suites
174~~~~~~~~~~~
175
176Now obviously one unit test isn't very helpful; the power comes from having
Brendan Higginse7d7ad02019-11-19 15:38:10 -0800177many test cases covering all of a unit's behaviors. Consequently it is common
178to have many *similar* tests; in order to reduce duplication in these closely
179related tests most unit testing frameworks - including KUnit - provide the
180concept of a *test suite*. A *test suite* is just a collection of test cases
181for a unit of code with a set up function that gets invoked before every test
182case and then a tear down function that gets invoked after every test case
183completes.
Brendan Higginsc23a2832019-09-23 02:02:45 -0700184
185Example:
186
187.. code-block:: c
188
189 static struct kunit_case example_test_cases[] = {
190 KUNIT_CASE(example_test_foo),
191 KUNIT_CASE(example_test_bar),
192 KUNIT_CASE(example_test_baz),
193 {}
194 };
195
196 static struct kunit_suite example_test_suite = {
197 .name = "example",
198 .init = example_test_init,
199 .exit = example_test_exit,
200 .test_cases = example_test_cases,
201 };
202 kunit_test_suite(example_test_suite);
203
204In the above example the test suite, ``example_test_suite``, would run the test
Randy Dunlap873ddeb2020-10-28 10:43:19 -0700205cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``;
Brendan Higginsc23a2832019-09-23 02:02:45 -0700206each would have ``example_test_init`` called immediately before it and would
207have ``example_test_exit`` called immediately after it.
208``kunit_test_suite(example_test_suite)`` registers the test suite with the
209KUnit test framework.
210
211.. note::
212 A test case will only be run if it is associated with a test suite.
213
Brendan Higginsa82763e2020-08-04 13:47:45 -0700214``kunit_test_suite(...)`` is a macro which tells the linker to put the specified
215test suite in a special linker section so that it can be run by KUnit either
216after late_init, or when the test module is loaded (depending on whether the
217test was built in or not).
218
Brendan Higginse7d7ad02019-11-19 15:38:10 -0800219For more information on these types of things see the :doc:`api/test`.
Brendan Higginsc23a2832019-09-23 02:02:45 -0700220
Daniel Latypov1f0e9432020-11-23 14:57:59 -0800221Common Patterns
222===============
223
Brendan Higginsc23a2832019-09-23 02:02:45 -0700224Isolating Behavior
Daniel Latypov1f0e9432020-11-23 14:57:59 -0800225------------------
Brendan Higginsc23a2832019-09-23 02:02:45 -0700226
227The most important aspect of unit testing that other forms of testing do not
228provide is the ability to limit the amount of code under test to a single unit.
229In practice, this is only possible by being able to control what code gets run
230when the unit under test calls a function and this is usually accomplished
231through some sort of indirection where a function is exposed as part of an API
232such that the definition of that function can be changed without affecting the
233rest of the code base. In the kernel this primarily comes from two constructs,
234classes, structs that contain function pointers that are provided by the
Randy Dunlap873ddeb2020-10-28 10:43:19 -0700235implementer, and architecture-specific functions which have definitions selected
Brendan Higginsc23a2832019-09-23 02:02:45 -0700236at compile time.
237
238Classes
Daniel Latypov1f0e9432020-11-23 14:57:59 -0800239~~~~~~~
Brendan Higginsc23a2832019-09-23 02:02:45 -0700240
241Classes are not a construct that is built into the C programming language;
242however, it is an easily derived concept. Accordingly, pretty much every project
243that does not use a standardized object oriented library (like GNOME's GObject)
244has their own slightly different way of doing object oriented programming; the
245Linux kernel is no exception.
246
247The central concept in kernel object oriented programming is the class. In the
248kernel, a *class* is a struct that contains function pointers. This creates a
249contract between *implementers* and *users* since it forces them to use the
250same function signature without having to call the function directly. In order
251for it to truly be a class, the function pointers must specify that a pointer
252to the class, known as a *class handle*, be one of the parameters; this makes
253it possible for the member functions (also known as *methods*) to have access
254to member variables (more commonly known as *fields*) allowing the same
255implementation to have multiple *instances*.
256
257Typically a class can be *overridden* by *child classes* by embedding the
258*parent class* in the child class. Then when a method provided by the child
259class is called, the child implementation knows that the pointer passed to it is
260of a parent contained within the child; because of this, the child can compute
261the pointer to itself because the pointer to the parent is always a fixed offset
262from the pointer to the child; this offset is the offset of the parent contained
263in the child struct. For example:
264
265.. code-block:: c
266
267 struct shape {
268 int (*area)(struct shape *this);
269 };
270
271 struct rectangle {
272 struct shape parent;
273 int length;
274 int width;
275 };
276
277 int rectangle_area(struct shape *this)
278 {
279 struct rectangle *self = container_of(this, struct shape, parent);
280
281 return self->length * self->width;
282 };
283
284 void rectangle_new(struct rectangle *self, int length, int width)
285 {
286 self->parent.area = rectangle_area;
287 self->length = length;
288 self->width = width;
289 }
290
291In this example (as in most kernel code) the operation of computing the pointer
292to the child from the pointer to the parent is done by ``container_of``.
293
294Faking Classes
295~~~~~~~~~~~~~~
296
297In order to unit test a piece of code that calls a method in a class, the
298behavior of the method must be controllable, otherwise the test ceases to be a
299unit test and becomes an integration test.
300
301A fake just provides an implementation of a piece of code that is different than
302what runs in a production instance, but behaves identically from the standpoint
303of the callers; this is usually done to replace a dependency that is hard to
304deal with, or is slow.
305
306A good example for this might be implementing a fake EEPROM that just stores the
307"contents" in an internal buffer. For example, let's assume we have a class that
308represents an EEPROM:
309
310.. code-block:: c
311
312 struct eeprom {
313 ssize_t (*read)(struct eeprom *this, size_t offset, char *buffer, size_t count);
314 ssize_t (*write)(struct eeprom *this, size_t offset, const char *buffer, size_t count);
315 };
316
317And we want to test some code that buffers writes to the EEPROM:
318
319.. code-block:: c
320
321 struct eeprom_buffer {
322 ssize_t (*write)(struct eeprom_buffer *this, const char *buffer, size_t count);
323 int flush(struct eeprom_buffer *this);
324 size_t flush_count; /* Flushes when buffer exceeds flush_count. */
325 };
326
327 struct eeprom_buffer *new_eeprom_buffer(struct eeprom *eeprom);
328 void destroy_eeprom_buffer(struct eeprom *eeprom);
329
330We can easily test this code by *faking out* the underlying EEPROM:
331
332.. code-block:: c
333
334 struct fake_eeprom {
335 struct eeprom parent;
336 char contents[FAKE_EEPROM_CONTENTS_SIZE];
337 };
338
339 ssize_t fake_eeprom_read(struct eeprom *parent, size_t offset, char *buffer, size_t count)
340 {
341 struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent);
342
343 count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset);
344 memcpy(buffer, this->contents + offset, count);
345
346 return count;
347 }
348
Brendan Higginse7d7ad02019-11-19 15:38:10 -0800349 ssize_t fake_eeprom_write(struct eeprom *parent, size_t offset, const char *buffer, size_t count)
Brendan Higginsc23a2832019-09-23 02:02:45 -0700350 {
351 struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent);
352
353 count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset);
354 memcpy(this->contents + offset, buffer, count);
355
356 return count;
357 }
358
359 void fake_eeprom_init(struct fake_eeprom *this)
360 {
361 this->parent.read = fake_eeprom_read;
362 this->parent.write = fake_eeprom_write;
363 memset(this->contents, 0, FAKE_EEPROM_CONTENTS_SIZE);
364 }
365
366We can now use it to test ``struct eeprom_buffer``:
367
368.. code-block:: c
369
370 struct eeprom_buffer_test {
371 struct fake_eeprom *fake_eeprom;
372 struct eeprom_buffer *eeprom_buffer;
373 };
374
375 static void eeprom_buffer_test_does_not_write_until_flush(struct kunit *test)
376 {
377 struct eeprom_buffer_test *ctx = test->priv;
378 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
379 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
380 char buffer[] = {0xff};
381
382 eeprom_buffer->flush_count = SIZE_MAX;
383
384 eeprom_buffer->write(eeprom_buffer, buffer, 1);
385 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
386
387 eeprom_buffer->write(eeprom_buffer, buffer, 1);
388 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0);
389
390 eeprom_buffer->flush(eeprom_buffer);
391 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
392 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
393 }
394
395 static void eeprom_buffer_test_flushes_after_flush_count_met(struct kunit *test)
396 {
397 struct eeprom_buffer_test *ctx = test->priv;
398 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
399 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
400 char buffer[] = {0xff};
401
402 eeprom_buffer->flush_count = 2;
403
404 eeprom_buffer->write(eeprom_buffer, buffer, 1);
405 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
406
407 eeprom_buffer->write(eeprom_buffer, buffer, 1);
408 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
409 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
410 }
411
412 static void eeprom_buffer_test_flushes_increments_of_flush_count(struct kunit *test)
413 {
414 struct eeprom_buffer_test *ctx = test->priv;
415 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
416 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
417 char buffer[] = {0xff, 0xff};
418
419 eeprom_buffer->flush_count = 2;
420
421 eeprom_buffer->write(eeprom_buffer, buffer, 1);
422 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
423
424 eeprom_buffer->write(eeprom_buffer, buffer, 2);
425 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
426 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
427 /* Should have only flushed the first two bytes. */
428 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[2], 0);
429 }
430
431 static int eeprom_buffer_test_init(struct kunit *test)
432 {
433 struct eeprom_buffer_test *ctx;
434
435 ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
436 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx);
437
438 ctx->fake_eeprom = kunit_kzalloc(test, sizeof(*ctx->fake_eeprom), GFP_KERNEL);
439 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx->fake_eeprom);
440 fake_eeprom_init(ctx->fake_eeprom);
441
442 ctx->eeprom_buffer = new_eeprom_buffer(&ctx->fake_eeprom->parent);
443 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx->eeprom_buffer);
444
445 test->priv = ctx;
446
447 return 0;
448 }
449
450 static void eeprom_buffer_test_exit(struct kunit *test)
451 {
452 struct eeprom_buffer_test *ctx = test->priv;
453
454 destroy_eeprom_buffer(ctx->eeprom_buffer);
455 }
456
Daniel Latypov1f0e9432020-11-23 14:57:59 -0800457Testing against multiple inputs
458-------------------------------
459
460Testing just a few inputs might not be enough to have confidence that the code
461works correctly, e.g. for a hash function.
462
463In such cases, it can be helpful to have a helper macro or function, e.g. this
464fictitious example for ``sha1sum(1)``
465
466.. code-block:: c
467
468 /* Note: the cast is to satisfy overly strict type-checking. */
469 #define TEST_SHA1(in, want) \
470 sha1sum(in, out); \
471 KUNIT_EXPECT_STREQ_MSG(test, (char *)out, want, "sha1sum(%s)", in);
472
473 char out[40];
474 TEST_SHA1("hello world", "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
475 TEST_SHA1("hello world!", "430ce34d020724ed75a196dfc2ad67c77772d169");
476
477
478Note the use of ``KUNIT_EXPECT_STREQ_MSG`` to give more context when it fails
479and make it easier to track down. (Yes, in this example, ``want`` is likely
480going to be unique enough on its own).
481
482The ``_MSG`` variants are even more useful when the same expectation is called
483multiple times (in a loop or helper function) and thus the line number isn't
484enough to identify what failed, like below.
485
486In some cases, it can be helpful to write a *table-driven test* instead, e.g.
487
488.. code-block:: c
489
490 int i;
491 char out[40];
492
493 struct sha1_test_case {
494 const char *str;
495 const char *sha1;
496 };
497
498 struct sha1_test_case cases[] = {
499 {
500 .str = "hello world",
501 .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
502 },
503 {
504 .str = "hello world!",
505 .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169",
506 },
507 };
508 for (i = 0; i < ARRAY_SIZE(cases); ++i) {
509 sha1sum(cases[i].str, out);
510 KUNIT_EXPECT_STREQ_MSG(test, (char *)out, cases[i].sha1,
511 "sha1sum(%s)", cases[i].str);
512 }
513
514
515There's more boilerplate involved, but it can:
516
517* be more readable when there are multiple inputs/outputs thanks to field names,
518
519 * E.g. see ``fs/ext4/inode-test.c`` for an example of both.
520* reduce duplication if test cases can be shared across multiple tests.
521
522 * E.g. if we wanted to also test ``sha256sum``, we could add a ``sha256``
523 field and reuse ``cases``.
524
Daniel Latypov8db50be2020-12-15 16:22:46 -0800525* be converted to a "parameterized test", see below.
526
527Parameterized Testing
528~~~~~~~~~~~~~~~~~~~~~
529
530The table-driven testing pattern is common enough that KUnit has special
531support for it.
532
533Reusing the same ``cases`` array from above, we can write the test as a
534"parameterized test" with the following.
535
536.. code-block:: c
537
538 // This is copy-pasted from above.
539 struct sha1_test_case {
540 const char *str;
541 const char *sha1;
542 };
543 struct sha1_test_case cases[] = {
544 {
545 .str = "hello world",
546 .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
547 },
548 {
549 .str = "hello world!",
550 .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169",
551 },
552 };
553
554 // Need a helper function to generate a name for each test case.
555 static void case_to_desc(const struct sha1_test_case *t, char *desc)
556 {
557 strcpy(desc, t->str);
558 }
559 // Creates `sha1_gen_params()` to iterate over `cases`.
560 KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc);
561
562 // Looks no different from a normal test.
563 static void sha1_test(struct kunit *test)
564 {
565 // This function can just contain the body of the for-loop.
566 // The former `cases[i]` is accessible under test->param_value.
567 char out[40];
568 struct sha1_test_case *test_param = (struct sha1_test_case *)(test->param_value);
569
570 sha1sum(test_param->str, out);
571 KUNIT_EXPECT_STREQ_MSG(test, (char *)out, test_param->sha1,
572 "sha1sum(%s)", test_param->str);
573 }
574
575 // Instead of KUNIT_CASE, we use KUNIT_CASE_PARAM and pass in the
576 // function declared by KUNIT_ARRAY_PARAM.
577 static struct kunit_case sha1_test_cases[] = {
578 KUNIT_CASE_PARAM(sha1_test, sha1_gen_params),
579 {}
580 };
581
Brendan Higginsc23a2832019-09-23 02:02:45 -0700582.. _kunit-on-non-uml:
583
584KUnit on non-UML architectures
585==============================
586
587By default KUnit uses UML as a way to provide dependencies for code under test.
588Under most circumstances KUnit's usage of UML should be treated as an
589implementation detail of how KUnit works under the hood. Nevertheless, there
Randy Dunlap873ddeb2020-10-28 10:43:19 -0700590are instances where being able to run architecture-specific code or test
Brendan Higginsc23a2832019-09-23 02:02:45 -0700591against real hardware is desirable. For these reasons KUnit supports running on
592other architectures.
593
594Running existing KUnit tests on non-UML architectures
595-----------------------------------------------------
596
597There are some special considerations when running existing KUnit tests on
598non-UML architectures:
599
600* Hardware may not be deterministic, so a test that always passes or fails
601 when run under UML may not always do so on real hardware.
602* Hardware and VM environments may not be hermetic. KUnit tries its best to
603 provide a hermetic environment to run tests; however, it cannot manage state
604 that it doesn't know about outside of the kernel. Consequently, tests that
605 may be hermetic on UML may not be hermetic on other architectures.
606* Some features and tooling may not be supported outside of UML.
607* Hardware and VMs are slower than UML.
608
609None of these are reasons not to run your KUnit tests on real hardware; they are
610only things to be aware of when doing so.
611
Brendan Higgins12ca7a82021-05-26 14:24:07 -0700612Currently, the KUnit Wrapper (``tools/testing/kunit/kunit.py``) (aka
613kunit_tool) only fully supports running tests inside of UML and QEMU; however,
614this is only due to our own time limitations as humans working on KUnit. It is
615entirely possible to support other emulators and even actual hardware, but for
616now QEMU and UML is what is fully supported within the KUnit Wrapper. Again, to
617be clear, this is just the Wrapper. The actualy KUnit tests and the KUnit
618library they are written in is fully architecture agnostic and can be used in
619virtually any setup, you just won't have the benefit of typing a single command
620out of the box and having everything magically work perfectly.
Brendan Higginsc23a2832019-09-23 02:02:45 -0700621
Brendan Higgins12ca7a82021-05-26 14:24:07 -0700622Again, all core KUnit framework features are fully supported on all
623architectures, and using them is straightforward: Most popular architectures
624are supported directly in the KUnit Wrapper via QEMU. Currently, supported
625architectures on QEMU include:
626
627* i386
628* x86_64
629* arm
630* arm64
631* alpha
632* powerpc
633* riscv
634* s390
635* sparc
636
637In order to run KUnit tests on one of these architectures via QEMU with the
638KUnit wrapper, all you need to do is specify the flags ``--arch`` and
639``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
640the default KUnit tests on ARM in the following manner (assuming we have an ARM
641toolchain installed):
642
643.. code-block:: bash
644
645 tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
646
647Alternatively, if you want to run your tests on real hardware or in some other
648emulation environment, all you need to do is to take your kunitconfig, your
649Kconfig options for the tests you would like to run, and merge them into
650whatever config your are using for your platform. That's it!
Brendan Higginsc23a2832019-09-23 02:02:45 -0700651
652For example, let's say you have the following kunitconfig:
653
654.. code-block:: none
655
656 CONFIG_KUNIT=y
657 CONFIG_KUNIT_EXAMPLE_TEST=y
658
659If you wanted to run this test on an x86 VM, you might add the following config
660options to your ``.config``:
661
662.. code-block:: none
663
664 CONFIG_KUNIT=y
665 CONFIG_KUNIT_EXAMPLE_TEST=y
666 CONFIG_SERIAL_8250=y
667 CONFIG_SERIAL_8250_CONSOLE=y
668
669All these new options do is enable support for a common serial console needed
670for logging.
671
672Next, you could build a kernel with these tests as follows:
673
674
675.. code-block:: bash
676
677 make ARCH=x86 olddefconfig
678 make ARCH=x86
679
680Once you have built a kernel, you could run it on QEMU as follows:
681
682.. code-block:: bash
683
684 qemu-system-x86_64 -enable-kvm \
685 -m 1024 \
686 -kernel arch/x86_64/boot/bzImage \
687 -append 'console=ttyS0' \
688 --nographic
689
690Interspersed in the kernel logs you might see the following:
691
692.. code-block:: none
693
694 TAP version 14
695 # Subtest: example
696 1..1
697 # example_simple_test: initializing
698 ok 1 - example_simple_test
699 ok 1 - example
700
701Congratulations, you just ran a KUnit test on the x86 architecture!
702
Alan Maguire6ae2bfd2020-01-06 22:28:23 +0000703In a similar manner, kunit and kunit tests can also be built as modules,
704so if you wanted to run tests in this way you might add the following config
705options to your ``.config``:
706
707.. code-block:: none
708
709 CONFIG_KUNIT=m
710 CONFIG_KUNIT_EXAMPLE_TEST=m
711
712Once the kernel is built and installed, a simple
713
714.. code-block:: bash
Brendan Higginse20d8e82020-01-31 16:01:02 -0800715
Alan Maguire6ae2bfd2020-01-06 22:28:23 +0000716 modprobe example-test
717
718...will run the tests.
719
SeongJae Parkf0b62032020-10-21 21:25:18 +0200720.. note::
721 Note that you should make sure your test depends on ``KUNIT=y`` in Kconfig
722 if the test does not support module build. Otherwise, it will trigger
723 compile errors if ``CONFIG_KUNIT`` is ``m``.
724
Brendan Higginsc23a2832019-09-23 02:02:45 -0700725Writing new tests for other architectures
726-----------------------------------------
727
728The first thing you must do is ask yourself whether it is necessary to write a
729KUnit test for a specific architecture, and then whether it is necessary to
730write that test for a particular piece of hardware. In general, writing a test
731that depends on having access to a particular piece of hardware or software (not
732included in the Linux source repo) should be avoided at all costs.
733
734Even if you only ever plan on running your KUnit test on your hardware
735configuration, other people may want to run your tests and may not have access
736to your hardware. If you write your test to run on UML, then anyone can run your
737tests without knowing anything about your particular setup, and you can still
738run your tests on your hardware setup just by compiling for your architecture.
739
740.. important::
741 Always prefer tests that run on UML to tests that only run under a particular
742 architecture, and always prefer tests that run under QEMU or another easy
Brendan Higginse7d7ad02019-11-19 15:38:10 -0800743 (and monetarily free) to obtain software environment to a specific piece of
Brendan Higginsc23a2832019-09-23 02:02:45 -0700744 hardware.
745
746Nevertheless, there are still valid reasons to write an architecture or hardware
747specific test: for example, you might want to test some code that really belongs
748in ``arch/some-arch/*``. Even so, try your best to write the test so that it
749does not depend on physical hardware: if some of your test cases don't need the
750hardware, only require the hardware for tests that actually need it.
751
752Now that you have narrowed down exactly what bits are hardware specific, the
753actual procedure for writing and running the tests is pretty much the same as
754writing normal KUnit tests. One special caveat is that you have to reset
755hardware state in between test cases; if this is not possible, you may only be
756able to run one test case per invocation.
757
Randy Dunlap873ddeb2020-10-28 10:43:19 -0700758.. TODO(brendanhiggins@google.com): Add an actual example of an architecture-
Brendan Higginsc23a2832019-09-23 02:02:45 -0700759 dependent KUnit test.
Alan Maguire32526902020-03-26 14:25:10 +0000760
761KUnit debugfs representation
762============================
763When kunit test suites are initialized, they create an associated directory
Lothar Rubuschc4714b02020-04-15 20:16:53 +0000764in ``/sys/kernel/debug/kunit/<test-suite>``. The directory contains one file
Alan Maguire32526902020-03-26 14:25:10 +0000765
766- results: "cat results" displays results of each test case and the results
767 of the entire suite for the last test run.
768
769The debugfs representation is primarily of use when kunit test suites are
770run in a native environment, either as modules or builtin. Having a way
771to display results like this is valuable as otherwise results can be
772intermixed with other events in dmesg output. The maximum size of each
Lothar Rubuschc4714b02020-04-15 20:16:53 +0000773results file is KUNIT_LOG_SIZE bytes (defined in ``include/kunit/test.h``).