[3/4] linux-dpdk: init: pass only one core through -c parameter

Message ID 1450285590-25226-4-git-send-email-zoltan.kiss@linaro.org
State New
Headers show

Commit Message

Zoltan Kiss Dec. 16, 2015, 5:06 p.m.
The other would be used for DPDK lcore threads, which are not accessible
for an ODP application.
Also reset the original affinity of the calling thread, as DPDK limits it
down to one core.

Signed-off-by: Zoltan Kiss <zoltan.kiss@linaro.org>
---
 platform/linux-dpdk/README     |  8 +++++++-
 platform/linux-dpdk/odp_init.c | 27 +++++++++++++++++++++++++--
 2 files changed, 32 insertions(+), 3 deletions(-)

Patch hide | download patch | download mbox

diff --git a/platform/linux-dpdk/README b/platform/linux-dpdk/README
index f351240..7f53f1d 100644
--- a/platform/linux-dpdk/README
+++ b/platform/linux-dpdk/README
@@ -234,7 +234,13 @@  variable:
 
 You need to pass "-n [1..4]" at least to specify memory channels. The coremask
 (-c) is calculated by ODP-DPDK based on the process affinity at startup. You can
-infcluence that with 'taskset'. 
+influence that with 'taskset'. DPDK init changes the affinity of the calling
+thread, so after it returns the original affinity is restored. Only the first
+active core is passed to rte_eal_init(), as the rest would be used for DPDK's
+special lcore threads, which are only available through
+rte_eal_[mp_]remote_launch(), but not through ODP API's. Nevertheless,
+odp_local_init() makes sure for the rest of the DPDK libraries ODP threads look
+like proper DPDK threads.
 Some useful ODP examples and how to run them:
 
     l2fwding app:
diff --git a/platform/linux-dpdk/odp_init.c b/platform/linux-dpdk/odp_init.c
index 8a8b785..b5b11c4 100644
--- a/platform/linux-dpdk/odp_init.c
+++ b/platform/linux-dpdk/odp_init.c
@@ -4,6 +4,11 @@ 
  * SPDX-License-Identifier:     BSD-3-Clause
  */
 
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <sched.h>
+
 #include <odp/api/cpu.h>
 #include <odp/init.h>
 #include <odp_internal.h>
@@ -141,6 +146,7 @@  int odp_init_dpdk(const char *cmdline)
 	odp_cpumask_t mask;
 	char mask_str[ODP_CPUMASK_STR_SIZE];
 	int32_t masklen;
+	cpu_set_t original_cpuset;
 
 	if (cmdline == NULL) {
 		cmdline = getenv("ODP_PLATFORM_PARAMS");
@@ -150,9 +156,21 @@  int odp_init_dpdk(const char *cmdline)
 		}
 	}
 
+	CPU_ZERO(&original_cpuset);
+	i = pthread_getaffinity_np(pthread_self(),
+				   sizeof(original_cpuset), &original_cpuset);
+	if (i != 0) {
+		ODP_ERR("Failed to read thread affinity: %d\n", i);
+		return -1;
+	}
+
 	odp_cpumask_zero(&mask);
-	for (i = 0; i <  odp_cpu_count(); i++)
-		odp_cpumask_set(&mask, i);
+	for (i = 0; i < CPU_SETSIZE; i++) {
+		if (CPU_ISSET(i, &original_cpuset)) {
+			odp_cpumask_set(&mask, i);
+			break;
+		}
+	}
 	masklen = odp_cpumask_to_str(&mask, mask_str, ODP_CPUMASK_STR_SIZE);
 
 	if (masklen < 0) {
@@ -196,6 +214,11 @@  int odp_init_dpdk(const char *cmdline)
 	}
 	ODP_DBG("rte_eal_init OK\n");
 
+	i = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t),
+				   &original_cpuset);
+	if (i)
+		ODP_ERR("Failed to reset thread affinity: %d\n", i);
+
 	return 0;
 }