Matt Fleming 668ba0
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Matt Fleming 668ba0
Date: Tue, 30 May 2017 16:39:01 +0200
Matt Fleming 668ba0
Subject: char/random: don't print that the init is done
Matt Fleming 668ba0
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git
Matt Fleming 668ba0
Git-commit: 4c441f8b036a13296cc41973523d6ed2bf3a7fe8
Matt Fleming 668ba0
Patch-mainline: Queued in subsystem maintainer repository
Matt Fleming 668ba0
References: SLE Realtime Extension
Matt Fleming 668ba0
Matt Fleming 668ba0
On RT we run into circular locking with pendingb_lock (workqueue),
Matt Fleming 668ba0
port_lock_key (uart) and the primary_crng (random):
Matt Fleming 668ba0
Matt Fleming 668ba0
 ======================================================
Matt Fleming 668ba0
 [ INFO: possible circular locking dependency detected ]
Matt Fleming 668ba0
 -------------------------------------------------------
Matt Fleming 668ba0
 irq/4-serial/512 is trying to acquire lock:
Matt Fleming 668ba0
  ((pendingb_lock).lock){+.+...}, at: [<ffffffff8108d48d>] queue_work_on+0x5d/0x190
Matt Fleming 668ba0
Matt Fleming 668ba0
 but task is already holding lock:
Matt Fleming 668ba0
  (&port_lock_key){+.+...}, at: [<ffffffff815b4bb6>] serial8250_handle_irq.part.27+0x16/0xb0
Matt Fleming 668ba0
Matt Fleming 668ba0
 which lock already depends on the new lock.
Matt Fleming 668ba0
Matt Fleming 668ba0
 the existing dependency chain (in reverse order) is:
Matt Fleming 668ba0
Matt Fleming 668ba0
 -> #3 (&port_lock_key){+.+...}:
Matt Fleming 668ba0
        lock_acquire+0xac/0x240
Matt Fleming 668ba0
        rt_spin_lock+0x46/0x50
Matt Fleming 668ba0
        serial8250_console_write+0x211/0x220
Matt Fleming 668ba0
        univ8250_console_write+0x1c/0x20
Matt Fleming 668ba0
        console_unlock+0x563/0x5c0
Matt Fleming 668ba0
        vprintk_emit+0x277/0x320
Matt Fleming 668ba0
        vprintk_default+0x1a/0x20
Matt Fleming 668ba0
        vprintk_func+0x20/0x80
Matt Fleming 668ba0
        printk+0x3e/0x46
Matt Fleming 668ba0
        crng_fast_load+0xde/0xe0
Matt Fleming 668ba0
        add_interrupt_randomness+0x16c/0x1a0
Matt Fleming 668ba0
        irq_thread+0x15c/0x1e0
Matt Fleming 668ba0
        kthread+0x112/0x150
Matt Fleming 668ba0
        ret_from_fork+0x31/0x40
Matt Fleming 668ba0
Matt Fleming 668ba0
 -> #2 (primary_crng.lock){+.+...}:
Matt Fleming 668ba0
        lock_acquire+0xac/0x240
Matt Fleming 668ba0
        rt_spin_lock+0x46/0x50
Matt Fleming 668ba0
        _extract_crng+0x39/0xa0
Matt Fleming 668ba0
        extract_crng+0x3a/0x40
Matt Fleming 668ba0
        get_random_u32+0x120/0x190
Matt Fleming 668ba0
        new_slab+0x1d6/0x7c0
Matt Fleming 668ba0
        ___slab_alloc+0x30b/0x6f0
Matt Fleming 668ba0
        __slab_alloc.isra.78+0x6c/0xc0
Matt Fleming 668ba0
        __kmalloc+0x254/0x3a0
Matt Fleming 668ba0
        pcpu_mem_zalloc+0x3a/0x70
Matt Fleming 668ba0
        percpu_init_late+0x4f/0x8a
Matt Fleming 668ba0
        start_kernel+0x1ec/0x3b8
Matt Fleming 668ba0
        x86_64_start_reservations+0x2a/0x2c
Matt Fleming 668ba0
        x86_64_start_kernel+0x13d/0x14c
Matt Fleming 668ba0
        verify_cpu+0x0/0xfc
Matt Fleming 668ba0
Matt Fleming 668ba0
 -> #1 ((batched_entropy_u32_lock).lock){+.+...}:
Matt Fleming 668ba0
        lock_acquire+0xac/0x240
Matt Fleming 668ba0
        rt_spin_lock__no_mg+0x41/0x50
Matt Fleming 668ba0
        get_random_u32+0x64/0x190
Matt Fleming 668ba0
        new_slab+0x1d6/0x7c0
Matt Fleming 668ba0
        ___slab_alloc+0x30b/0x6f0
Matt Fleming 668ba0
        __slab_alloc.isra.78+0x6c/0xc0
Matt Fleming 668ba0
        kmem_cache_alloc+0x26a/0x370
Matt Fleming 668ba0
        __debug_object_init+0x325/0x460
Matt Fleming 668ba0
        debug_object_activate+0x11c/0x1f0
Matt Fleming 668ba0
        __queue_work+0x2c/0x770
Matt Fleming 668ba0
        queue_work_on+0x12a/0x190
Matt Fleming 668ba0
        serio_queue_event+0xd3/0x140
Matt Fleming 668ba0
        __serio_register_port+0x17e/0x1a0
Matt Fleming 668ba0
        i8042_probe+0x623/0x687
Matt Fleming 668ba0
        platform_drv_probe+0x36/0x90
Matt Fleming 668ba0
        driver_probe_device+0x1f8/0x2e0
Matt Fleming 668ba0
        __driver_attach+0x96/0xa0
Matt Fleming 668ba0
        bus_for_each_dev+0x5d/0x90
Matt Fleming 668ba0
        driver_attach+0x19/0x20
Matt Fleming 668ba0
        bus_add_driver+0x125/0x220
Matt Fleming 668ba0
        driver_register+0x5b/0xd0
Matt Fleming 668ba0
        __platform_driver_probe+0x5b/0x120
Matt Fleming 668ba0
        __platform_create_bundle+0xaa/0xd0
Matt Fleming 668ba0
        i8042_init+0x3f1/0x430
Matt Fleming 668ba0
        do_one_initcall+0x3e/0x180
Matt Fleming 668ba0
        kernel_init_freeable+0x212/0x295
Matt Fleming 668ba0
        kernel_init+0x9/0x100
Matt Fleming 668ba0
        ret_from_fork+0x31/0x40
Matt Fleming 668ba0
Matt Fleming 668ba0
 -> #0 ((pendingb_lock).lock){+.+...}:
Matt Fleming 668ba0
        __lock_acquire+0x11b4/0x1320
Matt Fleming 668ba0
        lock_acquire+0xac/0x240
Matt Fleming 668ba0
        rt_spin_lock+0x46/0x50
Matt Fleming 668ba0
        queue_work_on+0x5d/0x190
Matt Fleming 668ba0
        tty_flip_buffer_push+0x26/0x30
Matt Fleming 668ba0
        serial8250_rx_chars+0x120/0x1f0
Matt Fleming 668ba0
        serial8250_handle_irq.part.27+0x58/0xb0
Matt Fleming 668ba0
        serial8250_default_handle_irq+0x4b/0x60
Matt Fleming 668ba0
        serial8250_interrupt+0x5f/0xd0
Matt Fleming 668ba0
        irq_forced_thread_fn+0x1e/0x70
Matt Fleming 668ba0
        irq_thread+0x137/0x1e0
Matt Fleming 668ba0
        kthread+0x112/0x150
Matt Fleming 668ba0
        ret_from_fork+0x31/0x40
Matt Fleming 668ba0
Matt Fleming 668ba0
 other info that might help us debug this:
Matt Fleming 668ba0
Matt Fleming 668ba0
 Chain exists of:
Matt Fleming 668ba0
   (pendingb_lock).lock --> primary_crng.lock --> &port_lock_key
Matt Fleming 668ba0
Matt Fleming 668ba0
  Possible unsafe locking scenario:
Matt Fleming 668ba0
Matt Fleming 668ba0
        CPU0                    CPU1
Matt Fleming 668ba0
        ----                    ----
Matt Fleming 668ba0
   lock(&port_lock_key);
Matt Fleming 668ba0
                                lock(primary_crng.lock);
Matt Fleming 668ba0
                                lock(&port_lock_key);
Matt Fleming 668ba0
   lock((pendingb_lock).lock);
Matt Fleming 668ba0
Matt Fleming 668ba0
  *** DEADLOCK ***
Matt Fleming 668ba0
Matt Fleming 668ba0
 2 locks held by irq/4-serial/512:
Matt Fleming 668ba0
  #0:  (&i->lock){+.+...}, at: [<ffffffff815b0400>] serial8250_interrupt+0x30/0xd0
Matt Fleming 668ba0
  #1:  (&port_lock_key){+.+...}, at: [<ffffffff815b4bb6>] serial8250_handle_irq.part.27+0x16/0xb0
Matt Fleming 668ba0
Matt Fleming 668ba0
 stack backtrace:
Matt Fleming 668ba0
 CPU: 4 PID: 512 Comm: irq/4-serial Not tainted 4.11.3-rt0+ #101
Matt Fleming 668ba0
 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.2-1 04/01/2014
Matt Fleming 668ba0
 Call Trace:
Matt Fleming 668ba0
  dump_stack+0x86/0xc1
Matt Fleming 668ba0
  print_circular_bug+0x1be/0x210
Matt Fleming 668ba0
  __lock_acquire+0x11b4/0x1320
Matt Fleming 668ba0
  lock_acquire+0xac/0x240
Matt Fleming 668ba0
  rt_spin_lock+0x46/0x50
Matt Fleming 668ba0
  queue_work_on+0x5d/0x190
Matt Fleming 668ba0
  tty_flip_buffer_push+0x26/0x30
Matt Fleming 668ba0
  serial8250_rx_chars+0x120/0x1f0
Matt Fleming 668ba0
  serial8250_handle_irq.part.27+0x58/0xb0
Matt Fleming 668ba0
  serial8250_default_handle_irq+0x4b/0x60
Matt Fleming 668ba0
  serial8250_interrupt+0x5f/0xd0
Matt Fleming 668ba0
  irq_forced_thread_fn+0x1e/0x70
Matt Fleming 668ba0
  irq_thread+0x137/0x1e0
Matt Fleming 668ba0
  kthread+0x112/0x150
Matt Fleming 668ba0
  ret_from_fork+0x31/0x40
Matt Fleming 668ba0
Matt Fleming 668ba0
It should work if we delay that printk after dropping the lock but we
Matt Fleming 668ba0
also could skip it.
Matt Fleming 668ba0
Matt Fleming 668ba0
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Matt Fleming 668ba0
Signed-off-by: Mike Galbraith <mgalbraith@suse.de>
Matt Fleming 668ba0
---
Matt Fleming 668ba0
 drivers/char/random.c |    2 --
Matt Fleming 668ba0
 1 file changed, 2 deletions(-)
Matt Fleming 668ba0
Matt Fleming 668ba0
--- a/drivers/char/random.c
Matt Fleming 668ba0
+++ b/drivers/char/random.c
Mel Gorman 888edd
@@ -864,7 +864,6 @@ static void crng_finalize_init(struct cr
Mel Gorman 888edd
 	crng_init = 2;
Mel Gorman 888edd
 	process_random_ready_list();
Mel Gorman 888edd
 	wake_up_interruptible(&crng_init_wait);
Mel Gorman 888edd
-	pr_notice("crng init done\n");
Mel Gorman 888edd
 	if (unseeded_warning.missed) {
Mel Gorman 888edd
 		pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
Mel Gorman 888edd
 			  unseeded_warning.missed);
Mel Gorman 888edd
@@ -901,7 +900,6 @@ static int crng_fast_load(const char *cp
Matt Fleming 668ba0
 	if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
Matt Fleming 668ba0
 		crng_init = 1;
Matt Fleming 668ba0
 		wake_up_interruptible(&crng_init_wait);
Matt Fleming 668ba0
-		pr_notice("random: fast init done\n");
Matt Fleming 668ba0
 	}
Matt Fleming 668ba0
 	return 1;
Matt Fleming 668ba0
 }