diff options
287 files changed, 10787 insertions, 2902 deletions
diff --git a/Documentation/netlink/specs/ethtool.yaml b/Documentation/netlink/specs/ethtool.yaml index 197208f419dc..7f6fb1f61dd4 100644 --- a/Documentation/netlink/specs/ethtool.yaml +++ b/Documentation/netlink/specs/ethtool.yaml @@ -16,6 +16,11 @@ definitions: name: stringset type: enum entries: [] + - + name: phy-upstream-type + enum-name: + type: enum + entries: [ mac, phy ] attribute-sets: - @@ -30,6 +35,9 @@ attribute-sets: - name: flags type: u32 + - + name: phy-index + type: u32 - name: bitset-bit @@ -942,6 +950,45 @@ attribute-sets: - name: burst-tmr type: u32 + - + name: phy-upstream + attributes: + - + name: index + type: u32 + - + name: sfp-name + type: string + - + name: phy + attributes: + - + name: header + type: nest + nested-attributes: header + - + name: index + type: u32 + - + name: drvname + type: string + - + name: name + type: string + - + name: upstream-type + type: u8 + enum: phy-upstream-type + - + name: upstream + type: nest + nested-attributes: phy-upstream + - + name: downstream-sfp-name + type: string + - + name: id + type: u32 operations: enum-model: directional @@ -1693,3 +1740,24 @@ operations: name: mm-ntf doc: Notification for change in MAC Merge configuration. notify: mm-get + - + name: phy-get + doc: Get PHY devices attached to an interface + + attribute-set: phy + + do: &phy-get-op + request: + attributes: + - header + reply: + attributes: + - header + - index + - drvname + - name + - upstream-type + - upstream + - downstream-sfp-name + - id + dump: *phy-get-op diff --git a/Documentation/networking/ethtool-netlink.rst b/Documentation/networking/ethtool-netlink.rst index d583d9abf2f8..97ff787a7dd8 100644 --- a/Documentation/networking/ethtool-netlink.rst +++ b/Documentation/networking/ethtool-netlink.rst @@ -57,6 +57,7 @@ Structure of this header is ``ETHTOOL_A_HEADER_DEV_INDEX`` u32 device ifindex ``ETHTOOL_A_HEADER_DEV_NAME`` string device name ``ETHTOOL_A_HEADER_FLAGS`` u32 flags common for all requests + ``ETHTOOL_A_HEADER_PHY_INDEX`` u32 phy device index ============================== ====== ============================= ``ETHTOOL_A_HEADER_DEV_INDEX`` and ``ETHTOOL_A_HEADER_DEV_NAME`` identify the @@ -81,6 +82,12 @@ the behaviour is backward compatible, i.e. requests from old clients not aware of the flag should be interpreted the way the client expects. A client must not set flags it does not understand. +``ETHTOOL_A_HEADER_PHY_INDEX`` identify the ethernet PHY the message relates to. +As there are numerous commands that are related to PHY configuration, and because +we can have more than one PHY on the link, the PHY index can be passed in the +request for the commands that needs it. It is however not mandatory, and if it +is not passed for commands that target a PHY, the net_device.phydev pointer +is used, as a fallback that keeps the legacy behaviour. Bit sets ======== @@ -2004,6 +2011,49 @@ The attributes are propagated to the driver through the following structure: .. kernel-doc:: include/linux/ethtool.h :identifiers: ethtool_mm_cfg +PHY_GET +======= + +Retrieve information about a given Ethernet PHY sitting on the link. As there +can be more than one PHY, the DUMP operation can be used to list the PHYs +present on a given interface, by passing an interface index or name in +the dump request + +Request contents: + + ==================================== ====== ========================== + ``ETHTOOL_A_PHY_HEADER`` nested request header + ==================================== ====== ========================== + +Kernel response contents: + + ===================================== ====== ========================== + ``ETHTOOL_A_PHY_HEADER`` nested request header + ``ETHTOOL_A_PHY_INDEX`` u32 the phy's unique index, that can + be used for phy-specific requests + ``ETHTOOL_A_PHY_DRVNAME`` string the phy driver name + ``ETHTOOL_A_PHY_NAME`` string the phy device name + ``ETHTOOL_A_PHY_UPSTREAM_TYPE`` u32 the type of device this phy is + connected to + ``ETHTOOL_A_PHY_UPSTREAM_PHY`` nested if the phy is connected to another + phy, this nest contains info on + that connection + ``ETHTOOL_A_PHY_DOWNSTREAM_SFP_NAME`` string if the phy controls an sfp bus, + the name of the sfp bus + ``ETHTOOL_A_PHY_ID`` u32 the phy id if the phy is C22 + ===================================== ====== ========================== + +When ``ETHTOOL_A_PHY_UPSTREAM_TYPE`` is PHY_UPSTREAM_PHY, the PHY's parent is +another PHY. Information on the parent PHY will be set in the +``ETHTOOL_A_PHY_UPSTREAM_PHY`` nest, which has the following structure : + + =================================== ====== ========================== + ``ETHTOOL_A_PHY_UPSTREAM_INDEX`` u32 the PHY index of the upstream PHY + ``ETHTOOL_A_PHY_UPSTREAM_SFP_NAME`` string if this PHY is connected to it's + parent PHY through an SFP bus, the + name of this sfp bus + =================================== ====== ========================== + Request translation =================== @@ -2110,4 +2160,5 @@ are netlink only. n/a ``ETHTOOL_MSG_PLCA_GET_STATUS`` n/a ``ETHTOOL_MSG_MM_GET`` n/a ``ETHTOOL_MSG_MM_SET`` + n/a ``ETHTOOL_MSG_PHY_GET`` =================================== ===================================== diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst index 69f3d6dcd9fd..a2c45a75a4a6 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -88,6 +88,7 @@ Contents: operstates packet_mmap phonet + phy-link-topology pktgen plip ppp_generic diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index 6cab1b797739..2dd8d8f20da2 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -38,7 +38,7 @@ netdev_features_t wanted_features netdev_features_t vlan_features netdev_features_t hw_enc_features - - netif_skb_features netdev_features_t mpls_features -netdev_features_t gso_partial_features +netdev_features_t gso_partial_features read_mostly gso_features_check unsigned_int min_mtu unsigned_int max_mtu unsigned_short type diff --git a/Documentation/networking/phy-link-topology.rst b/Documentation/networking/phy-link-topology.rst new file mode 100644 index 000000000000..1fd8e904ef4b --- /dev/null +++ b/Documentation/networking/phy-link-topology.rst @@ -0,0 +1,121 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +PHY link topology +================= + +Overview +======== + +The PHY link topology representation in the networking stack aims at representing +the hardware layout for any given Ethernet link. + +An Ethernet Interface from userspace's point of view is nothing but a +:c:type:`struct net_device <net_device>`, which exposes configuration options +through the legacy ioctls and the ethool netlink commands. The base assumption +when designing these configuration channels were that the link looked +something like this :: + + +-----------------------+ +----------+ +--------------+ + | Ethernet Controller / | | Ethernet | | Connector / | + | MAC | ------ | PHY | ---- | Port | ---... to LP + +-----------------------+ +----------+ +--------------+ + struct net_device struct phy_device + +Commands that needs to configure the PHY will go through the net_device.phydev +field to reach the PHY and perform the relevant configuration. + +This assumption falls apart in more complex topologies that can arise when, +for example, using SFP transceivers (although that's not the only specific case). + +Here, we have 2 basic scenarios. Either the MAC is able to output a serialized +interface, that can directly be fed to an SFP cage, such as SGMII, 1000BaseX, +10GBaseR, etc. + +The link topology then looks like this (when an SFP module is inserted) :: + + +-----+ SGMII +------------+ + | MAC | ------- | SFP Module | + +-----+ +------------+ + +Knowing that some modules embed a PHY, the actual link is more like :: + + +-----+ SGMII +--------------+ + | MAC | -------- | PHY (on SFP) | + +-----+ +--------------+ + +In this case, the SFP PHY is handled by phylib, and registered by phylink through +its SFP upstream ops. + +Now some Ethernet controllers aren't able to output a serialized interface, so +we can't directly connect them to an SFP cage. However, some PHYs can be used +as media-converters, to translate the non-serialized MAC MII interface to a +serialized MII interface fed to the SFP :: + + +-----+ RGMII +-----------------------+ SGMII +--------------+ + | MAC | ------- | PHY (media converter) | ------- | PHY (on SFP) | + +-----+ +-----------------------+ +--------------+ + +This is where the model of having a single net_device.phydev pointer shows its +limitations, as we now have 2 PHYs on the link. + +The phy_link topology framework aims at providing a way to keep track of every +PHY on the link, for use by both kernel drivers and subsystems, but also to +report the topology to userspace, allowing to target individual PHYs in configuration +commands. + +API +=== + +The :c:type:`struct phy_link_topology <phy_link_topology>` is a per-netdevice +resource, that gets initialized at netdevice creation. Once it's initialized, +it is then possible to register PHYs to the topology through : + +:c:func:`phy_link_topo_add_phy` + +Besides registering the PHY to the topology, this call will also assign a unique +index to the PHY, which can then be reported to userspace to refer to this PHY +(akin to the ifindex). This index is a u32, ranging from 1 to U32_MAX. The value +0 is reserved to indicate the PHY doesn't belong to any topology yet. + +The PHY can then be removed from the topology through + +:c:func:`phy_link_topo_del_phy` + +These function are already hooked into the phylib subsystem, so all PHYs that +are linked to a net_device through :c:func:`phy_attach_direct` will automatically +join the netdev's topology. + +PHYs that are on a SFP module will also be automatically registered IF the SFP +upstream is phylink (so, no media-converter). + +PHY drivers that can be used as SFP upstream need to call :c:func:`phy_sfp_attach_phy` +and :c:func:`phy_sfp_detach_phy`, which can be used as a +.attach_phy / .detach_phy implementation for the +:c:type:`struct sfp_upstream_ops <sfp_upstream_ops>`. + +UAPI +==== + +There exist a set of netlink commands to query the link topology from userspace, +see ``Documentation/networking/ethtool-netlink.rst``. + +The whole point of having a topology representation is to assign the phyindex +field in :c:type:`struct phy_device <phy_device>`. This index is reported to +userspace using the ``ETHTOOL_MSG_PHY_GET`` ethtnl command. Performing a DUMP operation +will result in all PHYs from all net_device being listed. The DUMP command +accepts either a ``ETHTOOL_A_HEADER_DEV_INDEX`` or ``ETHTOOL_A_HEADER_DEV_NAME`` +to be passed in the request to filter the DUMP to a single net_device. + +The retrieved index can then be passed as a request parameter using the +``ETHTOOL_A_HEADER_PHY_INDEX`` field in the following ethnl commands : + +* ``ETHTOOL_MSG_STRSET_GET`` to get the stats string set from a given PHY +* ``ETHTOOL_MSG_CABLE_TEST_ACT`` and ``ETHTOOL_MSG_CABLE_TEST_ACT``, to perform + cable testing on a given PHY on the link (most likely the outermost PHY) +* ``ETHTOOL_MSG_PSE_SET`` and ``ETHTOOL_MSG_PSE_GET`` for PHY-controlled PoE and PSE settings +* ``ETHTOOL_MSG_PLCA_GET_CFG``, ``ETHTOOL_MSG_PLCA_SET_CFG`` and ``ETHTOOL_MSG_PLCA_GET_STATUS`` + to set the PLCA (Physical Layer Collision Avoidance) parameters + +Note that the PHY index can be passed to other requests, which will silently +ignore it if present and irrelevant. diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt index 335b53df35e2..a8a1aff6445e 100644 --- a/Documentation/sphinx/requirements.txt +++ b/Documentation/sphinx/requirements.txt @@ -1,3 +1,4 @@ # jinja2>=3.1 is not compatible with Sphinx<4.0 jinja2<3.1 Sphinx==2.4.4 +pyyaml diff --git a/MAINTAINERS b/MAINTAINERS index fc044884c472..79ac49b113dc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7871,6 +7871,8 @@ F: include/linux/mii.h F: include/linux/of_net.h F: include/linux/phy.h F: include/linux/phy_fixed.h +F: include/linux/phy_link_topology.h +F: include/linux/phy_link_topology_core.h F: include/linux/phylib_stubs.h F: include/linux/platform_data/mdio-bcm-unimac.h F: include/linux/platform_data/mdio-gpio.h @@ -15444,7 +15446,7 @@ NXP C45 TJA11XX PHY DRIVER M: Radu Pirea <[email protected]> S: Maintained -F: drivers/net/phy/nxp-c45-tja11xx.c +F: drivers/net/phy/nxp-c45-tja11xx* NXP FSPI DRIVER M: Han Xu <[email protected]> diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig index 33795e4a5bd6..9c333d133c30 100644 --- a/arch/loongarch/configs/loongson3_defconfig +++ b/arch/loongarch/configs/loongson3_defconfig @@ -304,7 +304,6 @@ CONFIG_NET_CLS_ACT=y CONFIG_NET_ACT_POLICE=m CONFIG_NET_ACT_GACT=m CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_BPF=m CONFIG_OPENVSWITCH=m diff --git a/arch/mips/configs/ip22_defconfig b/arch/mips/configs/ip22_defconfig index dc49b09d492b..e22e8b825903 100644 --- a/arch/mips/configs/ip22_defconfig +++ b/arch/mips/configs/ip22_defconfig @@ -173,7 +173,6 @@ CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/mips/configs/malta_defconfig b/arch/mips/configs/malta_defconfig index 6f8046024557..4390d30206d9 100644 --- a/arch/mips/configs/malta_defconfig +++ b/arch/mips/configs/malta_defconfig @@ -202,7 +202,6 @@ CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/mips/configs/malta_kvm_defconfig b/arch/mips/configs/malta_kvm_defconfig index 16a91eeff67f..d63d8be8cb50 100644 --- a/arch/mips/configs/malta_kvm_defconfig +++ b/arch/mips/configs/malta_kvm_defconfig @@ -206,7 +206,6 @@ CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/mips/configs/maltaup_xpa_defconfig b/arch/mips/configs/maltaup_xpa_defconfig index 264aba29ea4f..338bb6544a93 100644 --- a/arch/mips/configs/maltaup_xpa_defconfig +++ b/arch/mips/configs/maltaup_xpa_defconfig @@ -203,7 +203,6 @@ CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/mips/configs/rb532_defconfig b/arch/mips/configs/rb532_defconfig index 02ec6c1a5116..517f1b060bf4 100644 --- a/arch/mips/configs/rb532_defconfig +++ b/arch/mips/configs/rb532_defconfig @@ -96,7 +96,6 @@ CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_PEDIT=m CONFIG_HAMRADIO=y CONFIG_MTD=y diff --git a/arch/powerpc/configs/ppc6xx_defconfig b/arch/powerpc/configs/ppc6xx_defconfig index f279703425d4..66c7b28d7450 100644 --- a/arch/powerpc/configs/ppc6xx_defconfig +++ b/arch/powerpc/configs/ppc6xx_defconfig @@ -274,7 +274,6 @@ CONFIG_NET_ACT_POLICE=m CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig index 6de44ede4e14..85490d9373fc 100644 --- a/arch/s390/configs/debug_defconfig +++ b/arch/s390/configs/debug_defconfig @@ -373,7 +373,6 @@ CONFIG_NET_ACT_POLICE=m CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig index bcae47da6b7c..fb690fbbf54b 100644 --- a/arch/s390/configs/defconfig +++ b/arch/s390/configs/defconfig @@ -363,7 +363,6 @@ CONFIG_NET_ACT_POLICE=m CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/arch/sh/configs/titan_defconfig b/arch/sh/configs/titan_defconfig index 871092753591..c1032559ecd4 100644 --- a/arch/sh/configs/titan_defconfig +++ b/arch/sh/configs/titan_defconfig @@ -138,7 +138,6 @@ CONFIG_NET_ACT_POLICE=m CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_PEDIT=m CONFIG_FW_LOADER=m CONFIG_CONNECTOR=m diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c index e01a246124ac..f3543a2df68d 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c @@ -289,7 +289,7 @@ static int dpaa2_switch_port_add_vlan(struct ethsw_port_priv *port_priv, int err; if (port_priv->vlans[vid]) { - netdev_warn(netdev, "VLAN %d already configured\n", vid); + netdev_err(netdev, "VLAN %d already configured\n", vid); return -EEXIST; } @@ -1509,9 +1509,9 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) struct device *dev = (struct device *)arg; struct ethsw_core *ethsw = dev_get_drvdata(dev); struct ethsw_port_priv *port_priv; - u32 status = ~0; int err, if_id; bool had_mac; + u32 status; err = dpsw_get_irq_status(ethsw->mc_io, 0, ethsw->dpsw_handle, DPSW_IRQ_INDEX_IF, &status); @@ -1523,12 +1523,11 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) if_id = (status & 0xFFFF0000) >> 16; port_priv = ethsw->ports[if_id]; - if (status & DPSW_IRQ_EVENT_LINK_CHANGED) { + if (status & DPSW_IRQ_EVENT_LINK_CHANGED) dpaa2_switch_port_link_state_update(port_priv->netdev); - dpaa2_switch_port_set_mac_addr(port_priv); - } if (status & DPSW_IRQ_EVENT_ENDPOINT_CHANGED) { + dpaa2_switch_port_set_mac_addr(port_priv); /* We can avoid locking because the "endpoint changed" IRQ * handler is the only one who changes priv->mac at runtime, * so we are not racing with anyone. @@ -1540,20 +1539,20 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) dpaa2_switch_port_connect_mac(port_priv); } -out: err = dpsw_clear_irq_status(ethsw->mc_io, 0, ethsw->dpsw_handle, DPSW_IRQ_INDEX_IF, status); if (err) dev_err(dev, "Can't clear irq status (err %d)\n", err); +out: return IRQ_HANDLED; } static int dpaa2_switch_setup_irqs(struct fsl_mc_device *sw_dev) { + u32 mask = DPSW_IRQ_EVENT_LINK_CHANGED | DPSW_IRQ_EVENT_ENDPOINT_CHANGED; struct device *dev = &sw_dev->dev; struct ethsw_core *ethsw = dev_get_drvdata(dev); - u32 mask = DPSW_IRQ_EVENT_LINK_CHANGED; struct fsl_mc_device_irq *irq; int err; @@ -1775,8 +1774,10 @@ int dpaa2_switch_port_vlans_add(struct net_device *netdev, /* Make sure that the VLAN is not already configured * on the switch port */ - if (port_priv->vlans[vlan->vid] & ETHSW_VLAN_MEMBER) + if (port_priv->vlans[vlan->vid] & ETHSW_VLAN_MEMBER) { + netdev_err(netdev, "VLAN %d already configured\n", vlan->vid); return -EEXIST; + } /* Check if there is space for a new VLAN */ err = dpsw_get_attributes(ethsw->mc_io, 0, ethsw->dpsw_handle, @@ -2003,25 +2004,11 @@ static int dpaa2_switch_port_bridge_join(struct net_device *netdev, struct netlink_ext_ack *extack) { struct ethsw_port_priv *port_priv = netdev_priv(netdev); + struct dpaa2_switch_fdb *old_fdb = port_priv->fdb; struct ethsw_core *ethsw = port_priv->ethsw_data; - struct ethsw_port_priv *other_port_priv; - struct net_device *other_dev; - struct list_head *iter; bool learn_ena; int err; - netdev_for_each_lower_dev(upper_dev, other_dev, iter) { - if (!dpaa2_switch_port_dev_check(other_dev)) - continue; - - other_port_priv = netdev_priv(other_dev); - if (other_port_priv->ethsw_data != port_priv->ethsw_data) { - NL_SET_ERR_MSG_MOD(extack, - "Interface from a different DPSW is in the bridge already"); - return -EINVAL; - } - } - /* Delete the previously manually installed VLAN 1 */ err = dpaa2_switch_port_del_vlan(port_priv, 1); if (err) @@ -2039,6 +2026,11 @@ static int dpaa2_switch_port_bridge_join(struct net_device *netdev, if (err) goto err_egress_flood; + /* Recreate the egress flood domain of the FDB that we just left. */ + err = dpaa2_switch_fdb_set_egress_flood(ethsw, old_fdb->fdb_id); + if (err) + goto err_egress_flood; + err = switchdev_bridge_port_offload(netdev, netdev, NULL, NULL, NULL, false, extack); if (err) @@ -2155,6 +2147,10 @@ dpaa2_switch_prechangeupper_sanity_checks(struct net_device *netdev, struct net_device *upper_dev, struct netlink_ext_ack *extack) { + struct ethsw_port_priv *port_priv = netdev_priv(netdev); + struct ethsw_port_priv *other_port_priv; + struct net_device *other_dev; + struct list_head *iter; int err; if (!br_vlan_enabled(upper_dev)) { @@ -2169,54 +2165,93 @@ dpaa2_switch_prechangeupper_sanity_checks(struct net_device *netdev, return 0; } + netdev_for_each_lower_dev(upper_dev, other_dev, iter) { + if (!dpaa2_switch_port_dev_check(other_dev)) + continue; + + other_port_priv = netdev_priv(other_dev); + if (other_port_priv->ethsw_data != port_priv->ethsw_data) { + NL_SET_ERR_MSG_MOD(extack, + "Interface from a different DPSW is in the bridge already"); + return -EINVAL; + } + } + return 0; } -static int dpaa2_switch_port_netdevice_event(struct notifier_block *nb, - unsigned long event, void *ptr) +static int dpaa2_switch_port_prechangeupper(struct net_device *netdev, + struct netdev_notifier_changeupper_info *info) { - struct net_device *netdev = netdev_notifier_info_to_dev(ptr); - struct netdev_notifier_changeupper_info *info = ptr; struct netlink_ext_ack *extack; struct net_device *upper_dev; - int err = 0; + int err; if (!dpaa2_switch_port_dev_check(netdev)) - return NOTIFY_DONE; + return 0; extack = netdev_notifier_info_to_extack(&info->info); - - switch (event) { - case NETDEV_PRECHANGEUPPER: - upper_dev = info->upper_dev; - if (!netif_is_bridge_master(upper_dev)) - break; - + upper_dev = info->upper_dev; + if (netif_is_bridge_master(upper_dev)) { err = dpaa2_switch_prechangeupper_sanity_checks(netdev, upper_dev, extack); if (err) - goto out; + return err; if (!info->linking) dpaa2_switch_port_pre_bridge_leave(netdev); + } + + return 0; +} + +static int dpaa2_switch_port_changeupper(struct net_device *netdev, + struct netdev_notifier_changeupper_info *info) +{ + struct netlink_ext_ack *extack; + struct net_device *upper_dev; + + if (!dpaa2_switch_port_dev_check(netdev)) + return 0; + + extack = netdev_notifier_info_to_extack(&info->info); + + upper_dev = info->upper_dev; + if (netif_is_bridge_master(upper_dev)) { + if (info->linking) + return dpaa2_switch_port_bridge_join(netdev, + upper_dev, + extack); + else + return dpaa2_switch_port_bridge_leave(netdev); + } + + return 0; +} + +static int dpaa2_switch_port_netdevice_event(struct notifier_block *nb, + unsigned long event, void *ptr) +{ + struct net_device *netdev = netdev_notifier_info_to_dev(ptr); + int err = 0; + + switch (event) { + case NETDEV_PRECHANGEUPPER: + err = dpaa2_switch_port_prechangeupper(netdev, ptr); + if (err) + return notifier_from_errno(err); break; case NETDEV_CHANGEUPPER: - upper_dev = info->upper_dev; - if (netif_is_bridge_master(upper_dev)) { - if (info->linking) - err = dpaa2_switch_port_bridge_join(netdev, - upper_dev, - extack); - else - err = dpaa2_switch_port_bridge_leave(netdev); - } + err = dpaa2_switch_port_changeupper(netdev, ptr); + if (err) + return notifier_from_errno(err); + break; } -out: - return notifier_from_errno(err); + return NOTIFY_DONE; } struct ethsw_switchdev_event_work { @@ -3294,6 +3329,7 @@ static int dpaa2_switch_probe_port(struct ethsw_core *ethsw, port_netdev->features = NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_VLAN_STAG_FILTER | NETIF_F_HW_TC; + port_netdev->priv_flags |= IFF_LIVE_ADDR_CHANGE; err = dpaa2_switch_port_init(port_priv, port_idx); if (err) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index c44870b175f9..f36232dead1a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -29,7 +29,7 @@ mlx5_core-$(CONFIG_MLX5_CORE_EN) += en/rqt.o en/tir.o en/rss.o en/rx_res.o \ en/reporter_tx.o en/reporter_rx.o en/params.o en/xsk/pool.o \ en/xsk/setup.o en/xsk/rx.o en/xsk/tx.o en/devlink.o en/ptp.o \ en/qos.o en/htb.o en/trap.o en/fs_tt_redirect.o en/selq.o \ - lib/crypto.o + en/mgmt_pf.o lib/crypto.o lib/sd.o # # Netdev extra diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index cf0477f53dc4..aa1b471e13fa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -190,6 +190,9 @@ bool mlx5_rdma_supported(struct mlx5_core_dev *dev) if (is_mp_supported(dev)) return false; + if (mlx5_core_is_mgmt_pf(dev)) + return false; + return true; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c index d000236ddbac..aa397e3ebe6d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c @@ -75,6 +75,9 @@ int mlx5_ec_init(struct mlx5_core_dev *dev) if (!mlx5_core_is_ecpf(dev)) return 0; + if (mlx5_core_is_mgmt_pf(dev)) + return 0; + return mlx5_host_pf_init(dev); } @@ -85,6 +88,9 @@ void mlx5_ec_cleanup(struct mlx5_core_dev *dev) if (!mlx5_core_is_ecpf(dev)) return; + if (mlx5_core_is_mgmt_pf(dev)) + return; + mlx5_host_pf_cleanup(dev); err = mlx5_wait_for_pages(dev, &dev->priv.page_counters[MLX5_HOST_PF]); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 0bfe1ca8a364..922b63c25154 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -60,8 +60,10 @@ #include "lib/clock.h" #include "en/rx_res.h" #include "en/selq.h" +#include "lib/sd.h" extern const struct net_device_ops mlx5e_netdev_ops; +extern const struct net_device_ops mlx5e_mgmt_netdev_ops; struct page_pool; #define MLX5E_METADATA_ETHER_TYPE (0x8CE4) @@ -791,6 +793,8 @@ struct mlx5e_channel { struct hwtstamp_config *tstamp; DECLARE_BITMAP(state, MLX5E_CHANNEL_NUM_STATES); int ix; + int vec_ix; + int sd_ix; int cpu; /* Sync between icosq recovery and XSK enable/disable. */ struct mutex icosq_recovery_lock; @@ -914,7 +918,7 @@ struct mlx5e_priv { bool tx_ptp_opened; bool rx_ptp_opened; struct hwtstamp_config tstamp; - u16 q_counter; + u16 q_counter[MLX5_SD_MAX_GROUP_SZ]; u16 drop_rq_q_counter; struct notifier_block events_nb; struct notifier_block blocking_events_nb; @@ -1029,12 +1033,12 @@ struct mlx5e_xsk_param; struct mlx5e_rq_param; int mlx5e_open_rq(struct mlx5e_params *params, struct mlx5e_rq_param *param, - struct mlx5e_xsk_param *xsk, int node, + struct mlx5e_xsk_param *xsk, int node, u16 q_counter, struct mlx5e_rq *rq); #define MLX5E_RQ_WQES_TIMEOUT 20000 /* msecs */ int mlx5e_wait_for_min_rx_wqes(struct mlx5e_rq *rq, int wait_time); void mlx5e_close_rq(struct mlx5e_rq *rq); -int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param); +int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param, u16 q_counter); void mlx5e_destroy_rq(struct mlx5e_rq *rq); struct mlx5e_sq_param; @@ -1122,9 +1126,10 @@ static inline bool mlx5_tx_swp_supported(struct mlx5_core_dev *mdev) } extern const struct ethtool_ops mlx5e_ethtool_ops; +extern const struct mlx5e_profile mlx5e_mgmt_pf_nic_profile; int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey); -int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev); +int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev, bool create_tises); void mlx5e_destroy_mdev_resources(struct mlx5_core_dev *mdev); int mlx5e_refresh_tirs(struct mlx5e_priv *priv, bool enable_uc_lb, bool enable_mc_lb); @@ -1227,6 +1232,8 @@ netdev_features_t mlx5e_features_check(struct sk_buff *skb, struct net_device *netdev, netdev_features_t features); int mlx5e_set_features(struct net_device *netdev, netdev_features_t features); +void mlx5e_nic_set_rx_mode(struct mlx5e_priv *priv); + #ifdef CONFIG_MLX5_ESWITCH int mlx5e_set_vf_mac(struct net_device *dev, int vf, u8 *mac); int mlx5e_set_vf_rate(struct net_device *dev, int vf, int min_tx_rate, int max_tx_rate); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c index 48581ea3adcb..874a1016623c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c @@ -23,20 +23,26 @@ bool mlx5e_channels_is_xsk(struct mlx5e_channels *chs, unsigned int ix) return test_bit(MLX5E_CHANNEL_STATE_XSK, c->state); } -void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) +void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn, + u32 *vhca_id) { struct mlx5e_channel *c = mlx5e_channels_get(chs, ix); *rqn = c->rq.rqn; + if (vhca_id) + *vhca_id = MLX5_CAP_GEN(c->mdev, vhca_id); } -void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) +void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn, + u32 *vhca_id) { struct mlx5e_channel *c = mlx5e_channels_get(chs, ix); WARN_ON_ONCE(!test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)); *rqn = c->xskrq.rqn; + if (vhca_id) + *vhca_id = MLX5_CAP_GEN(c->mdev, vhca_id); } bool mlx5e_channels_get_ptp_rqn(struct mlx5e_channels *chs, u32 *rqn) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h index 637ca90daaa8..6715aa9383b9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h @@ -10,8 +10,10 @@ struct mlx5e_channels; unsigned int mlx5e_channels_get_num(struct mlx5e_channels *chs); bool mlx5e_channels_is_xsk(struct mlx5e_channels *chs, unsigned int ix); -void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn); -void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn); +void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn, + u32 *vhca_id); +void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn, + u32 *vhca_id); bool mlx5e_channels_get_ptp_rqn(struct mlx5e_channels *chs, u32 *rqn); #endif /* __MLX5_EN_CHANNELS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/mgmt_pf.c b/drivers/net/ethernet/mellanox/mlx5/core/en/mgmt_pf.c new file mode 100644 index 000000000000..77b5805895b9 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/mgmt_pf.c @@ -0,0 +1,268 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +// Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +#include <linux/kernel.h> +#include "en/params.h" +#include "en/health.h" +#include "lib/eq.h" +#include "en/dcbnl.h" +#include "en_accel/ipsec.h" +#include "en_accel/en_accel.h" +#include "en/trap.h" +#include "en/monitor_stats.h" +#include "en/hv_vhca_stats.h" +#include "en_rep.h" +#include "en.h" + +static int mgmt_pf_async_event(struct notifier_block *nb, unsigned long event, void *data) +{ + struct mlx5e_priv *priv = container_of(nb, struct mlx5e_priv, events_nb); + struct mlx5_eqe *eqe = data; + + if (event != MLX5_EVENT_TYPE_PORT_CHANGE) + return NOTIFY_DONE; + + switch (eqe->sub_type) { + case MLX5_PORT_CHANGE_SUBTYPE_DOWN: + case MLX5_PORT_CHANGE_SUBTYPE_ACTIVE: + queue_work(priv->wq, &priv->update_carrier_work); + break; + default: + return NOTIFY_DONE; + } + + return NOTIFY_OK; +} + +static void mlx5e_mgmt_pf_enable_async_events(struct mlx5e_priv *priv) +{ + priv->events_nb.notifier_call = mgmt_pf_async_event; + mlx5_notifier_register(priv->mdev, &priv->events_nb); +} + +static void mlx5e_disable_mgmt_pf_async_events(struct mlx5e_priv *priv) +{ + mlx5_notifier_unregister(priv->mdev, &priv->events_nb); +} + +static void mlx5e_modify_mgmt_pf_admin_state(struct mlx5_core_dev *mdev, + enum mlx5_port_status state) +{ + struct mlx5_eswitch *esw = mdev->priv.eswitch; + int vport_admin_state; + + mlx5_set_port_admin_status(mdev, state); + + if (state == MLX5_PORT_UP) + vport_admin_state = MLX5_VPORT_ADMIN_STATE_AUTO; + else + vport_admin_state = MLX5_VPORT_ADMIN_STATE_DOWN; + + mlx5_eswitch_set_vport_state(esw, MLX5_VPORT_UPLINK, vport_admin_state); +} + +static void mlx5e_build_mgmt_pf_nic_params(struct mlx5e_priv *priv, u16 mtu) +{ + struct mlx5e_params *params = &priv->channels.params; + struct mlx5_core_dev *mdev = priv->mdev; + u8 rx_cq_period_mode; + + params->sw_mtu = mtu; + params->hard_mtu = MLX5E_ETH_HARD_MTU; + params->num_channels = 1; + + /* SQ */ + params->log_sq_size = is_kdump_kernel() ? + MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE : + MLX5E_PARAMS_DEFAULT_LOG_SQ_SIZE; + MLX5E_SET_PFLAG(params, MLX5E_PFLAG_SKB_TX_MPWQE, mlx5e_tx_mpwqe_supported(mdev)); + + MLX5E_SET_PFLAG(params, MLX5E_PFLAG_RX_NO_CSUM_COMPLETE, false); + + /* RQ */ + mlx5e_build_rq_params(mdev, params); + + /* CQ moderation params */ + rx_cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? + MLX5_CQ_PERIOD_MODE_START_FROM_CQE : + MLX5_CQ_PERIOD_MODE_START_FROM_EQE; + params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); + params->tx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); + mlx5e_set_rx_cq_mode_params(params, rx_cq_period_mode); + mlx5e_set_tx_cq_mode_params(params, MLX5_CQ_PERIOD_MODE_START_FROM_EQE); + + /* TX inline */ + mlx5_query_min_inline(mdev, ¶ms->tx_min_inline_mode); +} + +static int mlx5e_mgmt_pf_init(struct mlx5_core_dev *mdev, + struct net_device *netdev) +{ + struct mlx5e_priv *priv = netdev_priv(netdev); + struct mlx5e_flow_steering *fs; + int err; + + mlx5e_build_mgmt_pf_nic_params(priv, netdev->mtu); + + mlx5e_timestamp_init(priv); + + fs = mlx5e_fs_init(priv->profile, mdev, + !test_bit(MLX5E_STATE_DESTROYING, &priv->state), + priv->dfs_root); + if (!fs) { + err = -ENOMEM; + mlx5_core_err(mdev, "FS initialization failed, %d\n", err); + return err; + } + priv->fs = fs; + + mlx5e_health_create_reporters(priv); + + return 0; +} + +static void mlx5e_mgmt_pf_cleanup(struct mlx5e_priv *priv) +{ + mlx5e_health_destroy_reporters(priv); + mlx5e_fs_cleanup(priv->fs); + priv->fs = NULL; +} + +static int mlx5e_mgmt_pf_init_rx(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *mdev = priv->mdev; + int err; + + priv->rx_res = mlx5e_rx_res_create(mdev, 0, priv->max_nch, priv->drop_rq.rqn, + &priv->channels.params.packet_merge, + priv->channels.params.num_channels); + if (!priv->rx_res) + return -ENOMEM; + + mlx5e_create_q_counters(priv); + + err = mlx5e_open_drop_rq(priv, &priv->drop_rq); + if (err) { + mlx5_core_err(mdev, "open drop rq failed, %d\n", err); + goto err_destroy_q_counters; + } + + err = mlx5e_create_flow_steering(priv->fs, priv->rx_res, priv->profile, + priv->netdev); + if (err) { + mlx5_core_warn(mdev, "create flow steering failed, %d\n", err); + goto err_destroy_rx_res; + } + + return 0; + +err_destroy_rx_res: + mlx5e_rx_res_destroy(priv->rx_res); + priv->rx_res = NULL; + mlx5e_close_drop_rq(&priv->drop_rq); +err_destroy_q_counters: + mlx5e_destroy_q_counters(priv); + return err; +} + +static void mlx5e_mgmt_pf_cleanup_rx(struct mlx5e_priv *priv) +{ + mlx5e_destroy_flow_steering(priv->fs, !!(priv->netdev->hw_features & NETIF_F_NTUPLE), + priv->profile); + mlx5e_rx_res_destroy(priv->rx_res); + priv->rx_res = NULL; + mlx5e_close_drop_rq(&priv->drop_rq); + mlx5e_destroy_q_counters(priv); +} + +static int mlx5e_mgmt_pf_init_tx(struct mlx5e_priv *priv) +{ + return 0; +} + +static void mlx5e_mgmt_pf_cleanup_tx(struct mlx5e_priv *priv) +{ +} + +static void mlx5e_mgmt_pf_enable(struct mlx5e_priv *priv) +{ + struct net_device *netdev = priv->netdev; + struct mlx5_core_dev *mdev = priv->mdev; + + mlx5e_fs_init_l2_addr(priv->fs, netdev); + + /* Marking the link as currently not needed by the Driver */ + if (!netif_running(netdev)) + mlx5e_modify_mgmt_pf_admin_state(mdev, MLX5_PORT_DOWN); + + mlx5e_set_netdev_mtu_boundaries(priv); + mlx5e_set_dev_port_mtu(priv); + + mlx5e_mgmt_pf_enable_async_events(priv); + if (mlx5e_monitor_counter_supported(priv)) + mlx5e_monitor_counter_init(priv); + + mlx5e_hv_vhca_stats_create(priv); + if (netdev->reg_state != NETREG_REGISTERED) + return; + mlx5e_dcbnl_init_app(priv); + + mlx5e_nic_set_rx_mode(priv); + + rtnl_lock(); + if (netif_running(netdev)) + mlx5e_open(netdev); + udp_tunnel_nic_reset_ntf(priv->netdev); + netif_device_attach(netdev); + rtnl_unlock(); +} + +static void mlx5e_mgmt_pf_disable(struct mlx5e_priv *priv) +{ + if (priv->netdev->reg_state == NETREG_REGISTERED) + mlx5e_dcbnl_delete_app(priv); + + rtnl_lock(); + if (netif_running(priv->netdev)) + mlx5e_close(priv->netdev); + netif_device_detach(priv->netdev); + rtnl_unlock(); + + mlx5e_nic_set_rx_mode(priv); + + mlx5e_hv_vhca_stats_destroy(priv); + if (mlx5e_monitor_counter_supported(priv)) + mlx5e_monitor_counter_cleanup(priv); + + mlx5e_disable_mgmt_pf_async_events(priv); + mlx5e_ipsec_cleanup(priv); +} + +static int mlx5e_mgmt_pf_update_rx(struct mlx5e_priv *priv) +{ + return mlx5e_refresh_tirs(priv, false, false); +} + +static int mlx5e_mgmt_pf_max_nch_limit(struct mlx5_core_dev *mdev) +{ + return 1; +} + +const struct mlx5e_profile mlx5e_mgmt_pf_nic_profile = { + .init = mlx5e_mgmt_pf_init, + .cleanup = mlx5e_mgmt_pf_cleanup, + .init_rx = mlx5e_mgmt_pf_init_rx, + .cleanup_rx = mlx5e_mgmt_pf_cleanup_rx, + .init_tx = mlx5e_mgmt_pf_init_tx, + .cleanup_tx = mlx5e_mgmt_pf_cleanup_tx, + .enable = mlx5e_mgmt_pf_enable, + .disable = mlx5e_mgmt_pf_disable, + .update_rx = mlx5e_mgmt_pf_update_rx, + .update_stats = mlx5e_stats_update_ndo_stats, + .update_carrier = mlx5e_update_carrier, + .rx_handlers = &mlx5e_rx_handlers_nic, + .max_tc = 1, + .max_nch_limit = mlx5e_mgmt_pf_max_nch_limit, + .stats_grps = mlx5e_nic_stats_grps, + .stats_grps_num = mlx5e_nic_stats_grps_num +}; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/monitor_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en/monitor_stats.c index 40c8df111754..e2d8d2754be0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/monitor_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/monitor_stats.c @@ -20,10 +20,8 @@ #define NUM_REQ_PPCNT_COUNTER_S1 MLX5_CMD_SET_MONITOR_NUM_PPCNT_COUNTER_SET1 #define NUM_REQ_Q_COUNTERS_S1 MLX5_CMD_SET_MONITOR_NUM_Q_COUNTERS_SET1 -int mlx5e_monitor_counter_supported(struct mlx5e_priv *priv) +static int mlx5e_monitor_counter_cap(struct mlx5_core_dev *mdev) { - struct mlx5_core_dev *mdev = priv->mdev; - if (!MLX5_CAP_GEN(mdev, max_num_of_monitor_counters)) return false; if (MLX5_CAP_PCAM_REG(mdev, ppcnt) && @@ -36,24 +34,38 @@ int mlx5e_monitor_counter_supported(struct mlx5e_priv *priv) return true; } -static void mlx5e_monitor_counter_arm(struct mlx5e_priv *priv) +int mlx5e_monitor_counter_supported(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *pos; + int i; + + mlx5_sd_for_each_dev(i, priv->mdev, pos) + if (!mlx5e_monitor_counter_cap(pos)) + return false; + return true; +} + +static void mlx5e_monitor_counter_arm(struct mlx5_core_dev *mdev) { u32 in[MLX5_ST_SZ_DW(arm_monitor_counter_in)] = {}; MLX5_SET(arm_monitor_counter_in, in, opcode, MLX5_CMD_OP_ARM_MONITOR_COUNTER); - mlx5_cmd_exec_in(priv->mdev, arm_monitor_counter, in); + mlx5_cmd_exec_in(mdev, arm_monitor_counter, in); } static void mlx5e_monitor_counters_work(struct work_struct *work) { struct mlx5e_priv *priv = container_of(work, struct mlx5e_priv, monitor_counters_work); + struct mlx5_core_dev *pos; + int i; mutex_lock(&priv->state_lock); mlx5e_stats_update_ndo_stats(priv); mutex_unlock(&priv->state_lock); - mlx5e_monitor_counter_arm(priv); + mlx5_sd_for_each_dev(i, priv->mdev, pos) + mlx5e_monitor_counter_arm(pos); } static int mlx5e_monitor_event_handler(struct notifier_block *nb, @@ -97,15 +109,13 @@ static int fill_monitor_counter_q_counter_set1(int cnt, int q_counter, u32 *in) } /* check if mlx5e_monitor_counter_supported before calling this function*/ -static void mlx5e_set_monitor_counter(struct mlx5e_priv *priv) +static void mlx5e_set_monitor_counter(struct mlx5_core_dev *mdev, int q_counter) { - struct mlx5_core_dev *mdev = priv->mdev; int max_num_of_counters = MLX5_CAP_GEN(mdev, max_num_of_monitor_counters); int num_q_counters = MLX5_CAP_GEN(mdev, num_q_monitor_counters); int num_ppcnt_counters = !MLX5_CAP_PCAM_REG(mdev, ppcnt) ? 0 : MLX5_CAP_GEN(mdev, num_ppcnt_monitor_counters); u32 in[MLX5_ST_SZ_DW(set_monitor_counter_in)] = {}; - int q_counter = priv->q_counter; int cnt = 0; if (num_ppcnt_counters >= NUM_REQ_PPCNT_COUNTER_S1 && @@ -127,13 +137,17 @@ static void mlx5e_set_monitor_counter(struct mlx5e_priv *priv) /* check if mlx5e_monitor_counter_supported before calling this function*/ void mlx5e_monitor_counter_init(struct mlx5e_priv *priv) { + struct mlx5_core_dev *pos; + int i; + INIT_WORK(&priv->monitor_counters_work, mlx5e_monitor_counters_work); MLX5_NB_INIT(&priv->monitor_counters_nb, mlx5e_monitor_event_handler, MONITOR_COUNTER); - mlx5_eq_notifier_register(priv->mdev, &priv->monitor_counters_nb); - - mlx5e_set_monitor_counter(priv); - mlx5e_monitor_counter_arm(priv); + mlx5_sd_for_each_dev(i, priv->mdev, pos) { + mlx5_eq_notifier_register(pos, &priv->monitor_counters_nb); + mlx5e_set_monitor_counter(pos, priv->q_counter[i]); + mlx5e_monitor_counter_arm(pos); + } queue_work(priv->wq, &priv->update_stats_work); } @@ -141,11 +155,15 @@ void mlx5e_monitor_counter_init(struct mlx5e_priv *priv) void mlx5e_monitor_counter_cleanup(struct mlx5e_priv *priv) { u32 in[MLX5_ST_SZ_DW(set_monitor_counter_in)] = {}; + struct mlx5_core_dev *pos; + int i; MLX5_SET(set_monitor_counter_in, in, opcode, MLX5_CMD_OP_SET_MONITOR_COUNTER); - mlx5_cmd_exec_in(priv->mdev, set_monitor_counter, in); - mlx5_eq_notifier_unregister(priv->mdev, &priv->monitor_counters_nb); + mlx5_sd_for_each_dev(i, priv->mdev, pos) { + mlx5_cmd_exec_in(pos, set_monitor_counter, in); + mlx5_eq_notifier_unregister(pos, &priv->monitor_counters_nb); + } cancel_work_sync(&priv->monitor_counters_work); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c index 284253b79266..fb10bb166fbb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c @@ -674,7 +674,7 @@ void mlx5e_build_create_cq_param(struct mlx5e_create_cq_param *ccp, struct mlx5e .napi = &c->napi, .ch_stats = c->stats, .node = cpu_to_node(c->cpu), - .ix = c->ix, + .ix = c->vec_ix, }; } @@ -945,7 +945,6 @@ static u8 rq_end_pad_mode(struct mlx5_core_dev *mdev, struct mlx5e_params *param int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, - u16 q_counter, struct mlx5e_rq_param *param) { void *rqc = param->rqc; @@ -1007,7 +1006,6 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, MLX5_SET(wq, wq, log_wq_stride, mlx5e_get_rqwq_log_stride(params->rq_wq_type, ndsegs)); MLX5_SET(wq, wq, pd, mdev->mlx5e_res.hw_objs.pdn); - MLX5_SET(rqc, rqc, counter_set_id, q_counter); MLX5_SET(rqc, rqc, vsd, params->vlan_strip_disable); MLX5_SET(rqc, rqc, scatter_fcs, params->scatter_fcs_en); @@ -1018,7 +1016,6 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, } void mlx5e_build_drop_rq_param(struct mlx5_core_dev *mdev, - u16 q_counter, struct mlx5e_rq_param *param) { void *rqc = param->rqc; @@ -1027,7 +1024,6 @@ void mlx5e_build_drop_rq_param(struct mlx5_core_dev *mdev, MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_CYCLIC); MLX5_SET(wq, wq, log_wq_stride, mlx5e_get_rqwq_log_stride(MLX5_WQ_TYPE_CYCLIC, 1)); - MLX5_SET(rqc, rqc, counter_set_id, q_counter); param->wq.buf_numa_node = dev_to_node(mlx5_core_dma_dev(mdev)); } @@ -1292,13 +1288,12 @@ void mlx5e_build_xdpsq_param(struct mlx5_core_dev *mdev, int mlx5e_build_channel_param(struct mlx5_core_dev *mdev, struct mlx5e_params *params, - u16 q_counter, struct mlx5e_channel_param *cparam) { u8 icosq_log_wq_sz, async_icosq_log_wq_sz; int err; - err = mlx5e_build_rq_param(mdev, params, NULL, q_counter, &cparam->rq); + err = mlx5e_build_rq_param(mdev, params, NULL, &cparam->rq); if (err) return err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h index 6800949dafbc..9a781f18b57f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h @@ -130,10 +130,8 @@ void mlx5e_build_create_cq_param(struct mlx5e_create_cq_param *ccp, struct mlx5e int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, - u16 q_counter, struct mlx5e_rq_param *param); void mlx5e_build_drop_rq_param(struct mlx5_core_dev *mdev, - u16 q_counter, struct mlx5e_rq_param *param); void mlx5e_build_sq_param_common(struct mlx5_core_dev *mdev, struct mlx5e_sq_param *param); @@ -149,7 +147,6 @@ void mlx5e_build_xdpsq_param(struct mlx5_core_dev *mdev, struct mlx5e_sq_param *param); int mlx5e_build_channel_param(struct mlx5_core_dev *mdev, struct mlx5e_params *params, - u16 q_counter, struct mlx5e_channel_param *cparam); u16 mlx5e_calc_sq_stop_room(struct mlx5_core_dev *mdev, struct mlx5e_params *params); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c index c206cc0a8483..cafb41895f94 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c @@ -646,7 +646,6 @@ static void mlx5e_ptp_build_sq_param(struct mlx5_core_dev *mdev, static void mlx5e_ptp_build_rq_param(struct mlx5_core_dev *mdev, struct net_device *netdev, - u16 q_counter, struct mlx5e_ptp_params *ptp_params) { struct mlx5e_rq_param *rq_params = &ptp_params->rq_param; @@ -655,7 +654,7 @@ static void mlx5e_ptp_build_rq_param(struct mlx5_core_dev *mdev, params->rq_wq_type = MLX5_WQ_TYPE_CYCLIC; mlx5e_init_rq_type_params(mdev, params); params->sw_mtu = netdev->max_mtu; - mlx5e_build_rq_param(mdev, params, NULL, q_counter, rq_params); + mlx5e_build_rq_param(mdev, params, NULL, rq_params); } static void mlx5e_ptp_build_params(struct mlx5e_ptp *c, @@ -681,7 +680,7 @@ static void mlx5e_ptp_build_params(struct mlx5e_ptp *c, /* RQ */ if (test_bit(MLX5E_PTP_STATE_RX, c->state)) { params->vlan_strip_disable = orig->vlan_strip_disable; - mlx5e_ptp_build_rq_param(c->mdev, c->netdev, c->priv->q_counter, cparams); + mlx5e_ptp_build_rq_param(c->mdev, c->netdev, cparams); } } @@ -714,13 +713,16 @@ static int mlx5e_ptp_open_rq(struct mlx5e_ptp *c, struct mlx5e_params *params, struct mlx5e_rq_param *rq_param) { int node = dev_to_node(c->mdev->device); - int err; + int err, sd_ix; + u16 q_counter; err = mlx5e_init_ptp_rq(c, params, &c->rq); if (err) return err; - return mlx5e_open_rq(params, rq_param, NULL, node, &c->rq); + sd_ix = mlx5_sd_ch_ix_get_dev_ix(c->mdev, MLX5E_PTP_CHANNEL_IX); + q_counter = c->priv->q_counter[sd_ix]; + return mlx5e_open_rq(params, rq_param, NULL, node, q_counter, &c->rq); } static int mlx5e_ptp_open_queues(struct mlx5e_ptp *c, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c index 34adf8c3f81a..e87e26f2c669 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c @@ -122,8 +122,8 @@ int mlx5e_open_qos_sq(struct mlx5e_priv *priv, struct mlx5e_channels *chs, memset(¶m_sq, 0, sizeof(param_sq)); memset(¶m_cq, 0, sizeof(param_cq)); - mlx5e_build_sq_param(priv->mdev, params, ¶m_sq); - mlx5e_build_tx_cq_param(priv->mdev, params, ¶m_cq); + mlx5e_build_sq_param(c->mdev, params, ¶m_sq); + mlx5e_build_tx_cq_param(c->mdev, params, ¶m_cq); err = mlx5e_open_cq(c->mdev, params->tx_cq_moderation, ¶m_cq, &ccp, &sq->cq); if (err) goto err_free_sq; @@ -176,7 +176,7 @@ int mlx5e_activate_qos_sq(void *data, u16 node_qid, u32 hw_id) */ smp_wmb(); - qos_dbg(priv->mdev, "Activate QoS SQ qid %u\n", node_qid); + qos_dbg(sq->mdev, "Activate QoS SQ qid %u\n", node_qid); mlx5e_activate_txqsq(sq); return 0; @@ -190,7 +190,7 @@ void mlx5e_deactivate_qos_sq(struct mlx5e_priv *priv, u16 qid) if (!sq) /* Handle the case when the SQ failed to open. */ return; - qos_dbg(priv->mdev, "Deactivate QoS SQ qid %u\n", qid); + qos_dbg(sq->mdev, "Deactivate QoS SQ qid %u\n", qid); mlx5e_deactivate_txqsq(sq); priv->txq2sq[mlx5e_qid_from_qos(&priv->channels, qid)] = NULL; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c index 4358798d6ce1..25d751eba99b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c @@ -294,8 +294,8 @@ static void mlx5e_rx_reporter_diagnose_generic_rq(struct mlx5e_rq *rq, params = &priv->channels.params; rq_sz = mlx5e_rqwq_get_size(rq); - real_time = mlx5_is_real_time_rq(priv->mdev); - rq_stride = BIT(mlx5e_mpwqe_get_log_stride_size(priv->mdev, params, NULL)); + real_time = mlx5_is_real_time_rq(rq->mdev); + rq_stride = BIT(mlx5e_mpwqe_get_log_stride_size(rq->mdev, params, NULL)); mlx5e_health_fmsg_named_obj_nest_start(fmsg, "RQ"); devlink_fmsg_u8_pair_put(fmsg, "type", params->rq_wq_type); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c index 6b44ddce14e9..0ab9db319530 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c @@ -219,7 +219,6 @@ mlx5e_tx_reporter_build_diagnose_output_sq_common(struct devlink_fmsg *fmsg, struct mlx5e_txqsq *sq, int tc) { bool stopped = netif_xmit_stopped(sq->txq); - struct mlx5e_priv *priv = sq->priv; u8 state; int err; @@ -227,7 +226,7 @@ mlx5e_tx_reporter_build_diagnose_output_sq_common(struct devlink_fmsg *fmsg, devlink_fmsg_u32_pair_put(fmsg, "txq ix", sq->txq_ix); devlink_fmsg_u32_pair_put(fmsg, "sqn", sq->sqn); - err = mlx5_core_query_sq_state(priv->mdev, sq->sqn, &state); + err = mlx5_core_query_sq_state(sq->mdev, sq->sqn, &state); if (!err) devlink_fmsg_u8_pair_put(fmsg, "HW state", state); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c index 7b8ff7a71003..bcafb4bf9415 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c @@ -4,6 +4,33 @@ #include "rqt.h" #include <linux/mlx5/transobj.h> +static bool verify_num_vhca_ids(struct mlx5_core_dev *mdev, u32 *vhca_ids, + unsigned int size) +{ + unsigned int max_num_vhca_id = MLX5_CAP_GEN_2(mdev, max_rqt_vhca_id); + int i; + + /* Verify that all vhca_ids are in range [0, max_num_vhca_ids - 1] */ + for (i = 0; i < size; i++) + if (vhca_ids[i] >= max_num_vhca_id) + return false; + return true; +} + +static bool rqt_verify_vhca_ids(struct mlx5_core_dev *mdev, u32 *vhca_ids, + unsigned int size) +{ + if (!vhca_ids) + return true; + + if (!MLX5_CAP_GEN(mdev, cross_vhca_rqt)) + return false; + if (!verify_num_vhca_ids(mdev, vhca_ids, size)) + return false; + + return true; +} + void mlx5e_rss_params_indir_init_uniform(struct mlx5e_rss_params_indir *indir, unsigned int num_channels) { @@ -13,19 +40,38 @@ void mlx5e_rss_params_indir_init_uniform(struct mlx5e_rss_params_indir *indir, indir->table[i] = i % num_channels; } +static void fill_rqn_list(void *rqtc, u32 *rqns, u32 *vhca_ids, unsigned int size) +{ + unsigned int i; + + if (vhca_ids) { + MLX5_SET(rqtc, rqtc, rq_vhca_id_format, 1); + for (i = 0; i < size; i++) { + MLX5_SET(rqtc, rqtc, rq_vhca[i].rq_num, rqns[i]); + MLX5_SET(rqtc, rqtc, rq_vhca[i].rq_vhca_id, vhca_ids[i]); + } + } else { + for (i = 0; i < size; i++) + MLX5_SET(rqtc, rqtc, rq_num[i], rqns[i]); + } +} static int mlx5e_rqt_init(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, - u16 max_size, u32 *init_rqns, u16 init_size) + u16 max_size, u32 *init_rqns, u32 *init_vhca_ids, u16 init_size) { + int entry_sz; void *rqtc; int inlen; int err; u32 *in; - int i; + + if (!rqt_verify_vhca_ids(mdev, init_vhca_ids, init_size)) + return -EOPNOTSUPP; rqt->mdev = mdev; rqt->size = max_size; - inlen = MLX5_ST_SZ_BYTES(create_rqt_in) + sizeof(u32) * init_size; + entry_sz = init_vhca_ids ? MLX5_ST_SZ_BYTES(rq_vhca) : MLX5_ST_SZ_BYTES(rq_num); + inlen = MLX5_ST_SZ_BYTES(create_rqt_in) + entry_sz * init_size; in = kvzalloc(inlen, GFP_KERNEL); if (!in) return -ENOMEM; @@ -33,10 +79,9 @@ static int mlx5e_rqt_init(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, rqtc = MLX5_ADDR_OF(create_rqt_in, in, rqt_context); MLX5_SET(rqtc, rqtc, rqt_max_size, rqt->size); - MLX5_SET(rqtc, rqtc, rqt_actual_size, init_size); - for (i = 0; i < init_size; i++) - MLX5_SET(rqtc, rqtc, rq_num[i], init_rqns[i]); + + fill_rqn_list(rqtc, init_rqns, init_vhca_ids, init_size); err = mlx5_core_create_rqt(rqt->mdev, in, inlen, &rqt->rqtn); @@ -49,7 +94,7 @@ int mlx5e_rqt_init_direct(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, { u16 max_size = indir_enabled ? indir_table_size : 1; - return mlx5e_rqt_init(rqt, mdev, max_size, &init_rqn, 1); + return mlx5e_rqt_init(rqt, mdev, max_size, &init_rqn, NULL, 1); } static int mlx5e_bits_invert(unsigned long a, int size) @@ -63,7 +108,8 @@ static int mlx5e_bits_invert(unsigned long a, int size) return inv; } -static int mlx5e_calc_indir_rqns(u32 *rss_rqns, u32 *rqns, unsigned int num_rqns, +static int mlx5e_calc_indir_rqns(u32 *rss_rqns, u32 *rqns, u32 *rss_vhca_ids, u32 *vhca_ids, + unsigned int num_rqns, u8 hfunc, struct mlx5e_rss_params_indir *indir) { unsigned int i; @@ -82,30 +128,42 @@ static int mlx5e_calc_indir_rqns(u32 *rss_rqns, u32 *rqns, unsigned int num_rqns */ return -EINVAL; rss_rqns[i] = rqns[ix]; + if (vhca_ids) + rss_vhca_ids[i] = vhca_ids[ix]; } return 0; } int mlx5e_rqt_init_indir(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, - u32 *rqns, unsigned int num_rqns, + u32 *rqns, u32 *vhca_ids, unsigned int num_rqns, u8 hfunc, struct mlx5e_rss_params_indir *indir) { - u32 *rss_rqns; + u32 *rss_rqns, *rss_vhca_ids = NULL; int err; rss_rqns = kvmalloc_array(indir->actual_table_size, sizeof(*rss_rqns), GFP_KERNEL); if (!rss_rqns) return -ENOMEM; - err = mlx5e_calc_indir_rqns(rss_rqns, rqns, num_rqns, hfunc, indir); + if (vhca_ids) { + rss_vhca_ids = kvmalloc_array(indir->actual_table_size, sizeof(*rss_vhca_ids), + GFP_KERNEL); + if (!rss_vhca_ids) { + kvfree(rss_rqns); + return -ENOMEM; + } + } + + err = mlx5e_calc_indir_rqns(rss_rqns, rqns, rss_vhca_ids, vhca_ids, num_rqns, hfunc, indir); if (err) goto out; - err = mlx5e_rqt_init(rqt, mdev, indir->max_table_size, rss_rqns, + err = mlx5e_rqt_init(rqt, mdev, indir->max_table_size, rss_rqns, rss_vhca_ids, indir->actual_table_size); out: + kvfree(rss_vhca_ids); kvfree(rss_rqns); return err; } @@ -126,15 +184,20 @@ void mlx5e_rqt_destroy(struct mlx5e_rqt *rqt) mlx5_core_destroy_rqt(rqt->mdev, rqt->rqtn); } -static int mlx5e_rqt_redirect(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int size) +static int mlx5e_rqt_redirect(struct mlx5e_rqt *rqt, u32 *rqns, u32 *vhca_ids, + unsigned int size) { - unsigned int i; + int entry_sz; void *rqtc; int inlen; u32 *in; int err; - inlen = MLX5_ST_SZ_BYTES(modify_rqt_in) + sizeof(u32) * size; + if (!rqt_verify_vhca_ids(rqt->mdev, vhca_ids, size)) + return -EINVAL; + + entry_sz = vhca_ids ? MLX5_ST_SZ_BYTES(rq_vhca) : MLX5_ST_SZ_BYTES(rq_num); + inlen = MLX5_ST_SZ_BYTES(modify_rqt_in) + entry_sz * size; in = kvzalloc(inlen, GFP_KERNEL); if (!in) return -ENOMEM; @@ -143,8 +206,8 @@ static int mlx5e_rqt_redirect(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int siz MLX5_SET(modify_rqt_in, in, bitmask.rqn_list, 1); MLX5_SET(rqtc, rqtc, rqt_actual_size, size); - for (i = 0; i < size; i++) - MLX5_SET(rqtc, rqtc, rq_num[i], rqns[i]); + + fill_rqn_list(rqtc, rqns, vhca_ids, size); err = mlx5_core_modify_rqt(rqt->mdev, rqt->rqtn, in, inlen); @@ -152,17 +215,21 @@ static int mlx5e_rqt_redirect(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int siz return err; } -int mlx5e_rqt_redirect_direct(struct mlx5e_rqt *rqt, u32 rqn) +int mlx5e_rqt_redirect_direct(struct mlx5e_rqt *rqt, u32 rqn, u32 *vhca_id) { - return mlx5e_rqt_redirect(rqt, &rqn, 1); + return mlx5e_rqt_redirect(rqt, &rqn, vhca_id, 1); } -int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int num_rqns, +int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, u32 *vhca_ids, + unsigned int num_rqns, u8 hfunc, struct mlx5e_rss_params_indir *indir) { - u32 *rss_rqns; + u32 *rss_rqns, *rss_vhca_ids = NULL; int err; + if (!rqt_verify_vhca_ids(rqt->mdev, vhca_ids, num_rqns)) + return -EINVAL; + if (WARN_ON(rqt->size != indir->max_table_size)) return -EINVAL; @@ -170,13 +237,23 @@ int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int num_ if (!rss_rqns) return -ENOMEM; - err = mlx5e_calc_indir_rqns(rss_rqns, rqns, num_rqns, hfunc, indir); + if (vhca_ids) { + rss_vhca_ids = kvmalloc_array(indir->actual_table_size, sizeof(*rss_vhca_ids), + GFP_KERNEL); + if (!rss_vhca_ids) { + kvfree(rss_rqns); + return -ENOMEM; + } + } + + err = mlx5e_calc_indir_rqns(rss_rqns, rqns, rss_vhca_ids, vhca_ids, num_rqns, hfunc, indir); if (err) goto out; - err = mlx5e_rqt_redirect(rqt, rss_rqns, indir->actual_table_size); + err = mlx5e_rqt_redirect(rqt, rss_rqns, rss_vhca_ids, indir->actual_table_size); out: + kvfree(rss_vhca_ids); kvfree(rss_rqns); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h index 77fba3ebd18d..e0bc30308c77 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h @@ -20,7 +20,7 @@ void mlx5e_rss_params_indir_init_uniform(struct mlx5e_rss_params_indir *indir, unsigned int num_channels); struct mlx5e_rqt { - struct mlx5_core_dev *mdev; + struct mlx5_core_dev *mdev; /* primary */ u32 rqtn; u16 size; }; @@ -28,7 +28,7 @@ struct mlx5e_rqt { int mlx5e_rqt_init_direct(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, bool indir_enabled, u32 init_rqn, u32 indir_table_size); int mlx5e_rqt_init_indir(struct mlx5e_rqt *rqt, struct mlx5_core_dev *mdev, - u32 *rqns, unsigned int num_rqns, + u32 *rqns, u32 *vhca_ids, unsigned int num_rqns, u8 hfunc, struct mlx5e_rss_params_indir *indir); void mlx5e_rqt_destroy(struct mlx5e_rqt *rqt); @@ -38,8 +38,9 @@ static inline u32 mlx5e_rqt_get_rqtn(struct mlx5e_rqt *rqt) } u32 mlx5e_rqt_size(struct mlx5_core_dev *mdev, unsigned int num_channels); -int mlx5e_rqt_redirect_direct(struct mlx5e_rqt *rqt, u32 rqn); -int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, unsigned int num_rqns, +int mlx5e_rqt_redirect_direct(struct mlx5e_rqt *rqt, u32 rqn, u32 *vhca_id); +int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, u32 *vhca_ids, + unsigned int num_rqns, u8 hfunc, struct mlx5e_rss_params_indir *indir); #endif /* __MLX5_EN_RQT_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rss.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rss.c index c1545a2e8d6d..5f742f896600 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rss.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rss.c @@ -74,7 +74,7 @@ struct mlx5e_rss { struct mlx5e_tir *tir[MLX5E_NUM_INDIR_TIRS]; struct mlx5e_tir *inner_tir[MLX5E_NUM_INDIR_TIRS]; struct mlx5e_rqt rqt; - struct mlx5_core_dev *mdev; + struct mlx5_core_dev *mdev; /* primary */ u32 drop_rqn; bool inner_ft_support; bool enabled; @@ -473,21 +473,22 @@ int mlx5e_rss_obtain_tirn(struct mlx5e_rss *rss, return 0; } -static int mlx5e_rss_apply(struct mlx5e_rss *rss, u32 *rqns, unsigned int num_rqns) +static int mlx5e_rss_apply(struct mlx5e_rss *rss, u32 *rqns, u32 *vhca_ids, unsigned int num_rqns) { int err; - err = mlx5e_rqt_redirect_indir(&rss->rqt, rqns, num_rqns, rss->hash.hfunc, &rss->indir); + err = mlx5e_rqt_redirect_indir(&rss->rqt, rqns, vhca_ids, num_rqns, rss->hash.hfunc, + &rss->indir); if (err) mlx5e_rss_warn(rss->mdev, "Failed to redirect RQT %#x to channels: err = %d\n", mlx5e_rqt_get_rqtn(&rss->rqt), err); return err; } -void mlx5e_rss_enable(struct mlx5e_rss *rss, u32 *rqns, unsigned int num_rqns) +void mlx5e_rss_enable(struct mlx5e_rss *rss, u32 *rqns, u32 *vhca_ids, unsigned int num_rqns) { rss->enabled = true; - mlx5e_rss_apply(rss, rqns, num_rqns); + mlx5e_rss_apply(rss, rqns, vhca_ids, num_rqns); } void mlx5e_rss_disable(struct mlx5e_rss *rss) @@ -495,7 +496,7 @@ void mlx5e_rss_disable(struct mlx5e_rss *rss) int err; rss->enabled = false; - err = mlx5e_rqt_redirect_direct(&rss->rqt, rss->drop_rqn); + err = mlx5e_rqt_redirect_direct(&rss->rqt, rss->drop_rqn, NULL); if (err) mlx5e_rss_warn(rss->mdev, "Failed to redirect RQT %#x to drop RQ %#x: err = %d\n", mlx5e_rqt_get_rqtn(&rss->rqt), rss->drop_rqn, err); @@ -568,7 +569,7 @@ int mlx5e_rss_get_rxfh(struct mlx5e_rss *rss, u32 *indir, u8 *key, u8 *hfunc) int mlx5e_rss_set_rxfh(struct mlx5e_rss *rss, const u32 *indir, const u8 *key, const u8 *hfunc, - u32 *rqns, unsigned int num_rqns) + u32 *rqns, u32 *vhca_ids, unsigned int num_rqns) { bool changed_indir = false; bool changed_hash = false; @@ -608,7 +609,7 @@ int mlx5e_rss_set_rxfh(struct mlx5e_rss *rss, const u32 *indir, } if (changed_indir && rss->enabled) { - err = mlx5e_rss_apply(rss, rqns, num_rqns); + err = mlx5e_rss_apply(rss, rqns, vhca_ids, num_rqns); if (err) { mlx5e_rss_copy(rss, old_rss); goto out; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rss.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rss.h index d1d0bc350e92..d0df98963c8d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rss.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rss.h @@ -39,7 +39,7 @@ int mlx5e_rss_obtain_tirn(struct mlx5e_rss *rss, const struct mlx5e_packet_merge_param *init_pkt_merge_param, bool inner, u32 *tirn); -void mlx5e_rss_enable(struct mlx5e_rss *rss, u32 *rqns, unsigned int num_rqns); +void mlx5e_rss_enable(struct mlx5e_rss *rss, u32 *rqns, u32 *vhca_ids, unsigned int num_rqns); void mlx5e_rss_disable(struct mlx5e_rss *rss); int mlx5e_rss_packet_merge_set_param(struct mlx5e_rss *rss, @@ -47,7 +47,7 @@ int mlx5e_rss_packet_merge_set_param(struct mlx5e_rss *rss, int mlx5e_rss_get_rxfh(struct mlx5e_rss *rss, u32 *indir, u8 *key, u8 *hfunc); int mlx5e_rss_set_rxfh(struct mlx5e_rss *rss, const u32 *indir, const u8 *key, const u8 *hfunc, - u32 *rqns, unsigned int num_rqns); + u32 *rqns, u32 *vhca_ids, unsigned int num_rqns); struct mlx5e_rss_params_hash mlx5e_rss_get_hash(struct mlx5e_rss *rss); u8 mlx5e_rss_get_hash_fields(struct mlx5e_rss *rss, enum mlx5_traffic_types tt); int mlx5e_rss_set_hash_fields(struct mlx5e_rss *rss, enum mlx5_traffic_types tt, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c index b23e224e3763..a86eade9a9e0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c @@ -8,7 +8,7 @@ #define MLX5E_MAX_NUM_RSS 16 struct mlx5e_rx_res { - struct mlx5_core_dev *mdev; + struct mlx5_core_dev *mdev; /* primary */ enum mlx5e_rx_res_features features; unsigned int max_nch; u32 drop_rqn; @@ -19,6 +19,7 @@ struct mlx5e_rx_res { struct mlx5e_rss *rss[MLX5E_MAX_NUM_RSS]; bool rss_active; u32 *rss_rqns; + u32 *rss_vhca_ids; unsigned int rss_nch; struct { @@ -34,6 +35,13 @@ struct mlx5e_rx_res { /* API for rx_res_rss_* */ +static u32 *get_vhca_ids(struct mlx5e_rx_res *res, int offset) +{ + bool multi_vhca = res->features & MLX5E_RX_RES_FEATURE_MULTI_VHCA; + + return multi_vhca ? res->rss_vhca_ids + offset : NULL; +} + void mlx5e_rx_res_rss_update_num_channels(struct mlx5e_rx_res *res, u32 nch) { int i; @@ -85,8 +93,11 @@ int mlx5e_rx_res_rss_init(struct mlx5e_rx_res *res, u32 *rss_idx, unsigned int i return PTR_ERR(rss); mlx5e_rss_set_indir_uniform(rss, init_nch); - if (res->rss_active) - mlx5e_rss_enable(rss, res->rss_rqns, res->rss_nch); + if (res->rss_active) { + u32 *vhca_ids = get_vhca_ids(res, 0); + + mlx5e_rss_enable(rss, res->rss_rqns, vhca_ids, res->rss_nch); + } res->rss[i] = rss; *rss_idx = i; @@ -153,10 +164,12 @@ static void mlx5e_rx_res_rss_enable(struct mlx5e_rx_res *res) for (i = 0; i < MLX5E_MAX_NUM_RSS; i++) { struct mlx5e_rss *rss = res->rss[i]; + u32 *vhca_ids; if (!rss) continue; - mlx5e_rss_enable(rss, res->rss_rqns, res->rss_nch); + vhca_ids = get_vhca_ids(res, 0); + mlx5e_rss_enable(rss, res->rss_rqns, vhca_ids, res->rss_nch); } } @@ -200,6 +213,7 @@ int mlx5e_rx_res_rss_get_rxfh(struct mlx5e_rx_res *res, u32 rss_idx, int mlx5e_rx_res_rss_set_rxfh(struct mlx5e_rx_res *res, u32 rss_idx, const u32 *indir, const u8 *key, const u8 *hfunc) { + u32 *vhca_ids = get_vhca_ids(res, 0); struct mlx5e_rss *rss; if (rss_idx >= MLX5E_MAX_NUM_RSS) @@ -209,7 +223,8 @@ int mlx5e_rx_res_rss_set_rxfh(struct mlx5e_rx_res *res, u32 rss_idx, if (!rss) return -ENOENT; - return mlx5e_rss_set_rxfh(rss, indir, key, hfunc, res->rss_rqns, res->rss_nch); + return mlx5e_rss_set_rxfh(rss, indir, key, hfunc, res->rss_rqns, vhca_ids, + res->rss_nch); } int mlx5e_rx_res_rss_get_hash_fields(struct mlx5e_rx_res *res, u32 rss_idx, @@ -280,11 +295,13 @@ struct mlx5e_rss *mlx5e_rx_res_rss_get(struct mlx5e_rx_res *res, u32 rss_idx) static void mlx5e_rx_res_free(struct mlx5e_rx_res *res) { + kvfree(res->rss_vhca_ids); kvfree(res->rss_rqns); kvfree(res); } -static struct mlx5e_rx_res *mlx5e_rx_res_alloc(struct mlx5_core_dev *mdev, unsigned int max_nch) +static struct mlx5e_rx_res *mlx5e_rx_res_alloc(struct mlx5_core_dev *mdev, unsigned int max_nch, + bool multi_vhca) { struct mlx5e_rx_res *rx_res; @@ -298,6 +315,15 @@ static struct mlx5e_rx_res *mlx5e_rx_res_alloc(struct mlx5_core_dev *mdev, unsig return NULL; } + if (multi_vhca) { + rx_res->rss_vhca_ids = kvcalloc(max_nch, sizeof(*rx_res->rss_vhca_ids), GFP_KERNEL); + if (!rx_res->rss_vhca_ids) { + kvfree(rx_res->rss_rqns); + kvfree(rx_res); + return NULL; + } + } + return rx_res; } @@ -424,10 +450,11 @@ mlx5e_rx_res_create(struct mlx5_core_dev *mdev, enum mlx5e_rx_res_features featu const struct mlx5e_packet_merge_param *init_pkt_merge_param, unsigned int init_nch) { + bool multi_vhca = features & MLX5E_RX_RES_FEATURE_MULTI_VHCA; struct mlx5e_rx_res *res; int err; - res = mlx5e_rx_res_alloc(mdev, max_nch); + res = mlx5e_rx_res_alloc(mdev, max_nch, multi_vhca); if (!res) return ERR_PTR(-ENOMEM); @@ -504,10 +531,11 @@ static void mlx5e_rx_res_channel_activate_direct(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, unsigned int ix) { + u32 *vhca_id = get_vhca_ids(res, ix); u32 rqn = res->rss_rqns[ix]; int err; - err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, rqn); + err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, rqn, vhca_id); if (err) mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to RQ %#x (channel %u): err = %d\n", mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), @@ -519,7 +547,7 @@ static void mlx5e_rx_res_channel_deactivate_direct(struct mlx5e_rx_res *res, { int err; - err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, res->drop_rqn); + err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, res->drop_rqn, NULL); if (err) mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to drop RQ %#x (channel %u): err = %d\n", mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), @@ -534,10 +562,12 @@ void mlx5e_rx_res_channels_activate(struct mlx5e_rx_res *res, struct mlx5e_chann nch = mlx5e_channels_get_num(chs); for (ix = 0; ix < chs->num; ix++) { + u32 *vhca_id = get_vhca_ids(res, ix); + if (mlx5e_channels_is_xsk(chs, ix)) - mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix]); + mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix], vhca_id); else - mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix]); + mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix], vhca_id); } res->rss_nch = chs->num; @@ -554,7 +584,7 @@ void mlx5e_rx_res_channels_activate(struct mlx5e_rx_res *res, struct mlx5e_chann if (!mlx5e_channels_get_ptp_rqn(chs, &rqn)) rqn = res->drop_rqn; - err = mlx5e_rqt_redirect_direct(&res->ptp.rqt, rqn); + err = mlx5e_rqt_redirect_direct(&res->ptp.rqt, rqn, NULL); if (err) mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to RQ %#x (PTP): err = %d\n", mlx5e_rqt_get_rqtn(&res->ptp.rqt), @@ -573,7 +603,7 @@ void mlx5e_rx_res_channels_deactivate(struct mlx5e_rx_res *res) mlx5e_rx_res_channel_deactivate_direct(res, ix); if (res->features & MLX5E_RX_RES_FEATURE_PTP) { - err = mlx5e_rqt_redirect_direct(&res->ptp.rqt, res->drop_rqn); + err = mlx5e_rqt_redirect_direct(&res->ptp.rqt, res->drop_rqn, NULL); if (err) mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to drop RQ %#x (PTP): err = %d\n", mlx5e_rqt_get_rqtn(&res->ptp.rqt), @@ -584,10 +614,12 @@ void mlx5e_rx_res_channels_deactivate(struct mlx5e_rx_res *res) void mlx5e_rx_res_xsk_update(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, unsigned int ix, bool xsk) { + u32 *vhca_id = get_vhca_ids(res, ix); + if (xsk) - mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix]); + mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix], vhca_id); else - mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix]); + mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix], vhca_id); mlx5e_rx_res_rss_enable(res); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h index 82aaba8a82b3..7b1a9f0f1874 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h @@ -18,6 +18,7 @@ struct mlx5e_rss_params_hash; enum mlx5e_rx_res_features { MLX5E_RX_RES_FEATURE_INNER_FT = BIT(0), MLX5E_RX_RES_FEATURE_PTP = BIT(1), + MLX5E_RX_RES_FEATURE_MULTI_VHCA = BIT(2), }; /* Setup */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c b/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c index ac458a8d10e0..53ca16cb9c41 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c @@ -63,10 +63,12 @@ static int mlx5e_open_trap_rq(struct mlx5e_priv *priv, struct mlx5e_trap *t) struct mlx5e_create_cq_param ccp = {}; struct dim_cq_moder trap_moder = {}; struct mlx5e_rq *rq = &t->rq; + u16 q_counter; int node; int err; node = dev_to_node(mdev->device); + q_counter = priv->q_counter[0]; ccp.netdev = priv->netdev; ccp.wq = priv->wq; @@ -79,7 +81,7 @@ static int mlx5e_open_trap_rq(struct mlx5e_priv *priv, struct mlx5e_trap *t) return err; mlx5e_init_trap_rq(t, &t->params, rq); - err = mlx5e_open_rq(&t->params, rq_param, NULL, node, rq); + err = mlx5e_open_rq(&t->params, rq_param, NULL, node, q_counter, rq); if (err) goto err_destroy_cq; @@ -116,15 +118,14 @@ static int mlx5e_create_trap_direct_rq_tir(struct mlx5_core_dev *mdev, struct ml } static void mlx5e_build_trap_params(struct mlx5_core_dev *mdev, - int max_mtu, u16 q_counter, - struct mlx5e_trap *t) + int max_mtu, struct mlx5e_trap *t) { struct mlx5e_params *params = &t->params; params->rq_wq_type = MLX5_WQ_TYPE_CYCLIC; mlx5e_init_rq_type_params(mdev, params); params->sw_mtu = max_mtu; - mlx5e_build_rq_param(mdev, params, NULL, q_counter, &t->rq_param); + mlx5e_build_rq_param(mdev, params, NULL, &t->rq_param); } static struct mlx5e_trap *mlx5e_open_trap(struct mlx5e_priv *priv) @@ -138,7 +139,7 @@ static struct mlx5e_trap *mlx5e_open_trap(struct mlx5e_priv *priv) if (!t) return ERR_PTR(-ENOMEM); - mlx5e_build_trap_params(priv->mdev, netdev->max_mtu, priv->q_counter, t); + mlx5e_build_trap_params(priv->mdev, netdev->max_mtu, t); t->priv = priv; t->mdev = priv->mdev; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c index ebada0c5af3c..db776e515b6a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c @@ -6,10 +6,10 @@ #include "setup.h" #include "en/params.h" -static int mlx5e_xsk_map_pool(struct mlx5e_priv *priv, +static int mlx5e_xsk_map_pool(struct mlx5_core_dev *mdev, struct xsk_buff_pool *pool) { - struct device *dev = mlx5_core_dma_dev(priv->mdev); + struct device *dev = mlx5_core_dma_dev(mdev); return xsk_pool_dma_map(pool, dev, DMA_ATTR_SKIP_CPU_SYNC); } @@ -89,7 +89,7 @@ static int mlx5e_xsk_enable_locked(struct mlx5e_priv *priv, if (unlikely(!mlx5e_xsk_is_pool_sane(pool))) return -EINVAL; - err = mlx5e_xsk_map_pool(priv, pool); + err = mlx5e_xsk_map_pool(mlx5_sd_ch_ix_get_dev(priv->mdev, ix), pool); if (unlikely(err)) return err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c index 82e6abbc1734..06592b9f0424 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c @@ -49,10 +49,9 @@ bool mlx5e_validate_xsk_param(struct mlx5e_params *params, static void mlx5e_build_xsk_cparam(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, - u16 q_counter, struct mlx5e_channel_param *cparam) { - mlx5e_build_rq_param(mdev, params, xsk, q_counter, &cparam->rq); + mlx5e_build_rq_param(mdev, params, xsk, &cparam->rq); mlx5e_build_xdpsq_param(mdev, params, xsk, &cparam->xdp_sq); } @@ -93,6 +92,7 @@ static int mlx5e_open_xsk_rq(struct mlx5e_channel *c, struct mlx5e_params *param struct mlx5e_rq_param *rq_params, struct xsk_buff_pool *pool, struct mlx5e_xsk_param *xsk) { + u16 q_counter = c->priv->q_counter[c->sd_ix]; struct mlx5e_rq *xskrq = &c->xskrq; int err; @@ -100,7 +100,7 @@ static int mlx5e_open_xsk_rq(struct mlx5e_channel *c, struct mlx5e_params *param if (err) return err; - err = mlx5e_open_rq(params, rq_params, xsk, cpu_to_node(c->cpu), xskrq); + err = mlx5e_open_rq(params, rq_params, xsk, cpu_to_node(c->cpu), q_counter, xskrq); if (err) return err; @@ -125,7 +125,7 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, if (!cparam) return -ENOMEM; - mlx5e_build_xsk_cparam(priv->mdev, params, xsk, priv->q_counter, cparam); + mlx5e_build_xsk_cparam(priv->mdev, params, xsk, cparam); err = mlx5e_open_cq(c->mdev, params->rx_cq_moderation, &cparam->rq.cqp, &ccp, &c->xskrq.cq); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c index 984fa04bd331..e3e57c849436 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c @@ -96,7 +96,7 @@ bool mlx5e_is_ktls_rx(struct mlx5_core_dev *mdev) { u8 max_sq_wqebbs = mlx5e_get_max_sq_wqebbs(mdev); - if (is_kdump_kernel() || !MLX5_CAP_GEN(mdev, tls_rx)) + if (is_kdump_kernel() || !MLX5_CAP_GEN(mdev, tls_rx) || mlx5_get_sd(mdev)) return false; /* Check the possibility to post the required ICOSQ WQEs. */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h index f11075e67658..adc6d8ea0960 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h @@ -11,6 +11,7 @@ #ifdef CONFIG_MLX5_EN_TLS #include "lib/crypto.h" +#include "lib/mlx5.h" struct mlx5_crypto_dek *mlx5_ktls_create_key(struct mlx5_crypto_dek_pool *dek_pool, struct tls_crypto_info *crypto_info); @@ -61,7 +62,8 @@ void mlx5e_ktls_rx_resync_destroy_resp_list(struct mlx5e_ktls_resync_resp *resp_ static inline bool mlx5e_is_ktls_tx(struct mlx5_core_dev *mdev) { - return !is_kdump_kernel() && MLX5_CAP_GEN(mdev, tls_tx); + return !is_kdump_kernel() && MLX5_CAP_GEN(mdev, tls_tx) && + !mlx5_get_sd(mdev); } bool mlx5e_is_ktls_rx(struct mlx5_core_dev *mdev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c index 9b597cb24598..65ccb33edafb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c @@ -267,7 +267,7 @@ resync_post_get_progress_params(struct mlx5e_icosq *sq, goto err_out; } - pdev = mlx5_core_dma_dev(sq->channel->priv->mdev); + pdev = mlx5_core_dma_dev(sq->channel->mdev); buf->dma_addr = dma_map_single(pdev, &buf->progress, PROGRESS_PARAMS_PADDED_SIZE, DMA_FROM_DEVICE); if (unlikely(dma_mapping_error(pdev, buf->dma_addr))) { @@ -425,14 +425,12 @@ void mlx5e_ktls_handle_get_psv_completion(struct mlx5e_icosq_wqe_info *wi, { struct mlx5e_ktls_rx_resync_buf *buf = wi->tls_get_params.buf; struct mlx5e_ktls_offload_context_rx *priv_rx; - struct mlx5e_ktls_rx_resync_ctx *resync; u8 tracker_state, auth_state, *ctx; struct device *dev; u32 hw_seq; priv_rx = buf->priv_rx; - resync = &priv_rx->resync; - dev = mlx5_core_dma_dev(resync->priv->mdev); + dev = mlx5_core_dma_dev(sq->channel->mdev); if (unlikely(test_bit(MLX5E_PRIV_RX_FLAG_DELETING, priv_rx->flags))) goto out; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c index 67f546683e85..6ed3a32b7e22 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c @@ -95,7 +95,7 @@ static void mlx5e_destroy_tises(struct mlx5_core_dev *mdev, u32 tisn[MLX5_MAX_PO { int tc, i; - for (i = 0; i < MLX5_MAX_PORTS; i++) + for (i = 0; i < mlx5e_get_num_lag_ports(mdev); i++) for (tc = 0; tc < MLX5_MAX_NUM_TC; tc++) mlx5e_destroy_tis(mdev, tisn[i][tc]); } @@ -110,7 +110,7 @@ static int mlx5e_create_tises(struct mlx5_core_dev *mdev, u32 tisn[MLX5_MAX_PORT int tc, i; int err; - for (i = 0; i < MLX5_MAX_PORTS; i++) { + for (i = 0; i < mlx5e_get_num_lag_ports(mdev); i++) { for (tc = 0; tc < MLX5_MAX_NUM_TC; tc++) { u32 in[MLX5_ST_SZ_DW(create_tis_in)] = {}; void *tisc; @@ -140,7 +140,7 @@ err_close_tises: return err; } -int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev) +int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev, bool create_tises) { struct mlx5e_hw_objs *res = &mdev->mlx5e_res.hw_objs; int err; @@ -169,11 +169,15 @@ int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev) goto err_destroy_mkey; } - err = mlx5e_create_tises(mdev, res->tisn); - if (err) { - mlx5_core_err(mdev, "alloc tises failed, %d\n", err); - goto err_destroy_bfreg; + if (create_tises) { + err = mlx5e_create_tises(mdev, res->tisn); + if (err) { + mlx5_core_err(mdev, "alloc tises failed, %d\n", err); + goto err_destroy_bfreg; + } + res->tisn_valid = true; } + INIT_LIST_HEAD(&res->td.tirs_list); mutex_init(&res->td.list_lock); @@ -203,7 +207,8 @@ void mlx5e_destroy_mdev_resources(struct mlx5_core_dev *mdev) mlx5_crypto_dek_cleanup(mdev->mlx5e_res.dek_priv); mdev->mlx5e_res.dek_priv = NULL; - mlx5e_destroy_tises(mdev, res->tisn); + if (res->tisn_valid) + mlx5e_destroy_tises(mdev, res->tisn); mlx5_free_bfreg(mdev, &res->bfreg); mlx5_core_destroy_mkey(mdev, res->mkey); mlx5_core_dealloc_transport_domain(mdev, res->td.tdn); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index b5f1c4ca38ba..40626b6108fb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -70,6 +70,7 @@ #include "qos.h" #include "en/trap.h" #include "lib/devcom.h" +#include "lib/sd.h" bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift, enum mlx5e_mpwrq_umr_mode umr_mode) @@ -1024,7 +1025,7 @@ static void mlx5e_free_rq(struct mlx5e_rq *rq) mlx5_wq_destroy(&rq->wq_ctrl); } -int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param) +int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param, u16 q_counter) { struct mlx5_core_dev *mdev = rq->mdev; u8 ts_format; @@ -1051,6 +1052,7 @@ int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param) MLX5_SET(rqc, rqc, cqn, rq->cq.mcq.cqn); MLX5_SET(rqc, rqc, state, MLX5_RQC_STATE_RST); MLX5_SET(rqc, rqc, ts_format, ts_format); + MLX5_SET(rqc, rqc, counter_set_id, q_counter); MLX5_SET(wq, wq, log_wq_pg_sz, rq->wq_ctrl.buf.page_shift - MLX5_ADAPTER_PAGE_SHIFT); MLX5_SET64(wq, wq, dbr_addr, rq->wq_ctrl.db.dma); @@ -1274,7 +1276,7 @@ void mlx5e_free_rx_descs(struct mlx5e_rq *rq) } int mlx5e_open_rq(struct mlx5e_params *params, struct mlx5e_rq_param *param, - struct mlx5e_xsk_param *xsk, int node, + struct mlx5e_xsk_param *xsk, int node, u16 q_counter, struct mlx5e_rq *rq) { struct mlx5_core_dev *mdev = rq->mdev; @@ -1287,7 +1289,7 @@ int mlx5e_open_rq(struct mlx5e_params *params, struct mlx5e_rq_param *param, if (err) return err; - err = mlx5e_create_rq(rq, param); + err = mlx5e_create_rq(rq, param, q_counter); if (err) goto err_free_rq; @@ -2333,13 +2335,14 @@ static int mlx5e_set_tx_maxrate(struct net_device *dev, int index, u32 rate) static int mlx5e_open_rxq_rq(struct mlx5e_channel *c, struct mlx5e_params *params, struct mlx5e_rq_param *rq_params) { + u16 q_counter = c->priv->q_counter[c->sd_ix]; int err; err = mlx5e_init_rxq_rq(c, params, rq_params->xdp_frag_size, &c->rq); if (err) return err; - return mlx5e_open_rq(params, rq_params, NULL, cpu_to_node(c->cpu), &c->rq); + return mlx5e_open_rq(params, rq_params, NULL, cpu_to_node(c->cpu), q_counter, &c->rq); } static int mlx5e_open_queues(struct mlx5e_channel *c, @@ -2526,14 +2529,20 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, struct xsk_buff_pool *xsk_pool, struct mlx5e_channel **cp) { - int cpu = mlx5_comp_vector_get_cpu(priv->mdev, ix); struct net_device *netdev = priv->netdev; + struct mlx5_core_dev *mdev; struct mlx5e_xsk_param xsk; struct mlx5e_channel *c; unsigned int irq; + int vec_ix; + int cpu; int err; - err = mlx5_comp_irqn_get(priv->mdev, ix, &irq); + mdev = mlx5_sd_ch_ix_get_dev(priv->mdev, ix); + vec_ix = mlx5_sd_ch_ix_get_vec_ix(mdev, ix); + cpu = mlx5_comp_vector_get_cpu(mdev, vec_ix); + + err = mlx5_comp_irqn_get(mdev, vec_ix, &irq); if (err) return err; @@ -2546,18 +2555,20 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, return -ENOMEM; c->priv = priv; - c->mdev = priv->mdev; + c->mdev = mdev; c->tstamp = &priv->tstamp; c->ix = ix; + c->vec_ix = vec_ix; + c->sd_ix = mlx5_sd_ch_ix_get_dev_ix(mdev, ix); c->cpu = cpu; - c->pdev = mlx5_core_dma_dev(priv->mdev); + c->pdev = mlx5_core_dma_dev(mdev); c->netdev = priv->netdev; - c->mkey_be = cpu_to_be32(priv->mdev->mlx5e_res.hw_objs.mkey); + c->mkey_be = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey); c->num_tc = mlx5e_get_dcb_num_tc(params); c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix]->ch; c->aff_mask = irq_get_effective_affinity_mask(irq); - c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix); + c->lag_port = mlx5e_enumerate_lag_port(mdev, ix); netif_napi_add(netdev, &c->napi, mlx5e_napi_poll); @@ -2647,7 +2658,7 @@ int mlx5e_open_channels(struct mlx5e_priv *priv, if (!chs->c || !cparam) goto err_free; - err = mlx5e_build_channel_param(priv->mdev, &chs->params, priv->q_counter, cparam); + err = mlx5e_build_channel_param(priv->mdev, &chs->params, cparam); if (err) goto err_free; @@ -2935,15 +2946,18 @@ static MLX5E_DEFINE_PREACTIVATE_WRAPPER_CTX(mlx5e_update_netdev_queues); static void mlx5e_set_default_xps_cpumasks(struct mlx5e_priv *priv, struct mlx5e_params *params) { - struct mlx5_core_dev *mdev = priv->mdev; - int num_comp_vectors, ix, irq; - - num_comp_vectors = mlx5_comp_vectors_max(mdev); + int ix; for (ix = 0; ix < params->num_channels; ix++) { + int num_comp_vectors, irq, vec_ix; + struct mlx5_core_dev *mdev; + + mdev = mlx5_sd_ch_ix_get_dev(priv->mdev, ix); + num_comp_vectors = mlx5_comp_vectors_max(mdev); cpumask_clear(priv->scratchpad.cpumask); + vec_ix = mlx5_sd_ch_ix_get_vec_ix(mdev, ix); - for (irq = ix; irq < num_comp_vectors; irq += params->num_channels) { + for (irq = vec_ix; irq < num_comp_vectors; irq += params->num_channels) { int cpu = mlx5_comp_vector_get_cpu(mdev, irq); cpumask_set_cpu(cpu, priv->scratchpad.cpumask); @@ -3335,7 +3349,7 @@ int mlx5e_open_drop_rq(struct mlx5e_priv *priv, struct mlx5e_cq *cq = &drop_rq->cq; int err; - mlx5e_build_drop_rq_param(mdev, priv->drop_rq_q_counter, &rq_param); + mlx5e_build_drop_rq_param(mdev, &rq_param); err = mlx5e_alloc_drop_cq(priv, cq, &cq_param); if (err) @@ -3349,7 +3363,7 @@ int mlx5e_open_drop_rq(struct mlx5e_priv *priv, if (err) goto err_destroy_cq; - err = mlx5e_create_rq(drop_rq, &rq_param); + err = mlx5e_create_rq(drop_rq, &rq_param, priv->drop_rq_q_counter); if (err) goto err_free_rq; @@ -3785,7 +3799,7 @@ mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats) stats->tx_errors = stats->tx_aborted_errors + stats->tx_carrier_errors; } -static void mlx5e_nic_set_rx_mode(struct mlx5e_priv *priv) +void mlx5e_nic_set_rx_mode(struct mlx5e_priv *priv) { if (mlx5e_is_uplink_rep(priv)) return; /* no rx mode for uplink rep */ @@ -4990,6 +5004,15 @@ const struct net_device_ops mlx5e_netdev_ops = { #endif }; +const struct net_device_ops mlx5e_mgmt_netdev_ops = { + .ndo_open = mlx5e_open, + .ndo_stop = mlx5e_close, + .ndo_start_xmit = mlx5e_xmit, + .ndo_get_stats64 = mlx5e_get_stats, + .ndo_change_mtu = mlx5e_change_nic_mtu, + .ndo_set_rx_mode = mlx5e_set_rx_mode, +}; + static u32 mlx5e_choose_lro_timeout(struct mlx5_core_dev *mdev, u32 wanted_timeout) { int i; @@ -5129,7 +5152,11 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) SET_NETDEV_DEV(netdev, mdev->device); - netdev->netdev_ops = &mlx5e_netdev_ops; + if (mlx5_core_is_mgmt_pf(mdev)) + netdev->netdev_ops = &mlx5e_mgmt_netdev_ops; + else + netdev->netdev_ops = &mlx5e_netdev_ops; + netdev->xdp_metadata_ops = &mlx5e_xdp_metadata_ops; netdev->xsk_tx_metadata_ops = &mlx5e_xsk_tx_metadata_ops; @@ -5264,13 +5291,17 @@ void mlx5e_create_q_counters(struct mlx5e_priv *priv) u32 out[MLX5_ST_SZ_DW(alloc_q_counter_out)] = {}; u32 in[MLX5_ST_SZ_DW(alloc_q_counter_in)] = {}; struct mlx5_core_dev *mdev = priv->mdev; - int err; + struct mlx5_core_dev *pos; + int err, i; MLX5_SET(alloc_q_counter_in, in, opcode, MLX5_CMD_OP_ALLOC_Q_COUNTER); - err = mlx5_cmd_exec_inout(mdev, alloc_q_counter, in, out); - if (!err) - priv->q_counter = - MLX5_GET(alloc_q_counter_out, out, counter_set_id); + + mlx5_sd_for_each_dev(i, mdev, pos) { + err = mlx5_cmd_exec_inout(pos, alloc_q_counter, in, out); + if (!err) + priv->q_counter[i] = + MLX5_GET(alloc_q_counter_out, out, counter_set_id); + } err = mlx5_cmd_exec_inout(mdev, alloc_q_counter, in, out); if (!err) @@ -5281,13 +5312,17 @@ void mlx5e_create_q_counters(struct mlx5e_priv *priv) void mlx5e_destroy_q_counters(struct mlx5e_priv *priv) { u32 in[MLX5_ST_SZ_DW(dealloc_q_counter_in)] = {}; + struct mlx5_core_dev *pos; + int i; MLX5_SET(dealloc_q_counter_in, in, opcode, MLX5_CMD_OP_DEALLOC_Q_COUNTER); - if (priv->q_counter) { - MLX5_SET(dealloc_q_counter_in, in, counter_set_id, - priv->q_counter); - mlx5_cmd_exec_in(priv->mdev, dealloc_q_counter, in); + mlx5_sd_for_each_dev(i, priv->mdev, pos) { + if (priv->q_counter[i]) { + MLX5_SET(dealloc_q_counter_in, in, counter_set_id, + priv->q_counter[i]); + mlx5_cmd_exec_in(pos, dealloc_q_counter, in); + } } if (priv->drop_rq_q_counter) { @@ -5371,6 +5406,8 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) features = MLX5E_RX_RES_FEATURE_PTP; if (mlx5_tunnel_inner_ft_supported(mdev)) features |= MLX5E_RX_RES_FEATURE_INNER_FT; + if (mlx5_get_sd(priv->mdev)) + features |= MLX5E_RX_RES_FEATURE_MULTI_VHCA; priv->rx_res = mlx5e_rx_res_create(priv->mdev, features, priv->max_nch, priv->drop_rq.rqn, &priv->channels.params.packet_merge, @@ -5980,28 +6017,52 @@ void mlx5e_destroy_netdev(struct mlx5e_priv *priv) free_netdev(netdev); } -static int mlx5e_resume(struct auxiliary_device *adev) +static int _mlx5e_resume(struct auxiliary_device *adev) { struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); struct mlx5e_dev *mlx5e_dev = auxiliary_get_drvdata(adev); struct mlx5e_priv *priv = mlx5e_dev->priv; struct net_device *netdev = priv->netdev; struct mlx5_core_dev *mdev = edev->mdev; - int err; + struct mlx5_core_dev *pos, *to; + int err, i; if (netif_device_present(netdev)) return 0; - err = mlx5e_create_mdev_resources(mdev); - if (err) - return err; + mlx5_sd_for_each_dev(i, mdev, pos) { + err = mlx5e_create_mdev_resources(pos, true); + if (err) + goto err_destroy_mdev_res; + } err = mlx5e_attach_netdev(priv); - if (err) { - mlx5e_destroy_mdev_resources(mdev); + if (err) + goto err_destroy_mdev_res; + + return 0; + +err_destroy_mdev_res: + to = pos; + mlx5_sd_for_each_dev_to(i, mdev, to, pos) + mlx5e_destroy_mdev_resources(pos); + return err; +} + +static int mlx5e_resume(struct auxiliary_device *adev) +{ + struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); + struct mlx5_core_dev *mdev = edev->mdev; + struct auxiliary_device *actual_adev; + int err; + + err = mlx5_sd_init(mdev); + if (err) return err; - } + actual_adev = mlx5_sd_get_adev(mdev, adev, edev->idx); + if (actual_adev) + return _mlx5e_resume(actual_adev); return 0; } @@ -6011,33 +6072,53 @@ static int _mlx5e_suspend(struct auxiliary_device *adev) struct mlx5e_priv *priv = mlx5e_dev->priv; struct net_device *netdev = priv->netdev; struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5_core_dev *pos; + int i; if (!netif_device_present(netdev)) { if (test_bit(MLX5E_STATE_DESTROYING, &priv->state)) - mlx5e_destroy_mdev_resources(mdev); + mlx5_sd_for_each_dev(i, mdev, pos) + mlx5e_destroy_mdev_resources(pos); return -ENODEV; } mlx5e_detach_netdev(priv); - mlx5e_destroy_mdev_resources(mdev); + mlx5_sd_for_each_dev(i, mdev, pos) + mlx5e_destroy_mdev_resources(pos); + return 0; } static int mlx5e_suspend(struct auxiliary_device *adev, pm_message_t state) { - return _mlx5e_suspend(adev); + struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); + struct mlx5_core_dev *mdev = edev->mdev; + struct auxiliary_device *actual_adev; + int err = 0; + + actual_adev = mlx5_sd_get_adev(mdev, adev, edev->idx); + if (actual_adev) + err = _mlx5e_suspend(actual_adev); + + mlx5_sd_cleanup(mdev); + return err; } static int _mlx5e_probe(struct auxiliary_device *adev) { struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); - const struct mlx5e_profile *profile = &mlx5e_nic_profile; struct mlx5_core_dev *mdev = edev->mdev; + const struct mlx5e_profile *profile; struct mlx5e_dev *mlx5e_dev; struct net_device *netdev; struct mlx5e_priv *priv; int err; + if (mlx5_core_is_mgmt_pf(mdev)) + profile = &mlx5e_mgmt_pf_nic_profile; + else + profile = &mlx5e_nic_profile; + mlx5e_dev = mlx5e_create_devlink(&adev->dev, mdev); if (IS_ERR(mlx5e_dev)) return PTR_ERR(mlx5e_dev); @@ -6071,9 +6152,9 @@ static int _mlx5e_probe(struct auxiliary_device *adev) goto err_destroy_netdev; } - err = mlx5e_resume(adev); + err = _mlx5e_resume(adev); if (err) { - mlx5_core_err(mdev, "mlx5e_resume failed, %d\n", err); + mlx5_core_err(mdev, "_mlx5e_resume failed, %d\n", err); goto err_profile_cleanup; } @@ -6104,15 +6185,29 @@ err_devlink_unregister: static int mlx5e_probe(struct auxiliary_device *adev, const struct auxiliary_device_id *id) { - return _mlx5e_probe(adev); + struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); + struct mlx5_core_dev *mdev = edev->mdev; + struct auxiliary_device *actual_adev; + int err; + + err = mlx5_sd_init(mdev); + if (err) + return err; + + actual_adev = mlx5_sd_get_adev(mdev, adev, edev->idx); + if (actual_adev) + return _mlx5e_probe(actual_adev); + return 0; } -static void mlx5e_remove(struct auxiliary_device *adev) +static void _mlx5e_remove(struct auxiliary_device *adev) { + struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); struct mlx5e_dev *mlx5e_dev = auxiliary_get_drvdata(adev); struct mlx5e_priv *priv = mlx5e_dev->priv; + struct mlx5_core_dev *mdev = edev->mdev; - mlx5_core_uplink_netdev_set(priv->mdev, NULL); + mlx5_core_uplink_netdev_set(mdev, NULL); mlx5e_dcbnl_delete_app(priv); unregister_netdev(priv->netdev); _mlx5e_suspend(adev); @@ -6122,6 +6217,19 @@ static void mlx5e_remove(struct auxiliary_device *adev) mlx5e_destroy_devlink(mlx5e_dev); } +static void mlx5e_remove(struct auxiliary_device *adev) +{ + struct mlx5_adev *edev = container_of(adev, struct mlx5_adev, adev); + struct mlx5_core_dev *mdev = edev->mdev; + struct auxiliary_device *actual_adev; + + actual_adev = mlx5_sd_get_adev(mdev, adev, edev->idx); + if (actual_adev) + _mlx5e_remove(actual_adev); + + mlx5_sd_cleanup(mdev); +} + static const struct auxiliary_device_id mlx5e_id_table[] = { { .name = MLX5_ADEV_NAME ".eth", }, {}, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c index 4b96ad657145..f3d0898bdbc6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c @@ -561,11 +561,23 @@ static const struct counter_desc drop_rq_stats_desc[] = { #define NUM_Q_COUNTERS ARRAY_SIZE(q_stats_desc) #define NUM_DROP_RQ_COUNTERS ARRAY_SIZE(drop_rq_stats_desc) +static bool q_counter_any(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *pos; + int i; + + mlx5_sd_for_each_dev(i, priv->mdev, pos) + if (priv->q_counter[i++]) + return true; + + return false; +} + static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(qcnt) { int num_stats = 0; - if (priv->q_counter) + if (q_counter_any(priv)) num_stats += NUM_Q_COUNTERS; if (priv->drop_rq_q_counter) @@ -578,7 +590,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(qcnt) { int i; - for (i = 0; i < NUM_Q_COUNTERS && priv->q_counter; i++) + for (i = 0; i < NUM_Q_COUNTERS && q_counter_any(priv); i++) strcpy(data + (idx++) * ETH_GSTRING_LEN, q_stats_desc[i].format); @@ -593,7 +605,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(qcnt) { int i; - for (i = 0; i < NUM_Q_COUNTERS && priv->q_counter; i++) + for (i = 0; i < NUM_Q_COUNTERS && q_counter_any(priv); i++) data[idx++] = MLX5E_READ_CTR32_CPU(&priv->stats.qcnt, q_stats_desc, i); for (i = 0; i < NUM_DROP_RQ_COUNTERS && priv->drop_rq_q_counter; i++) @@ -607,18 +619,23 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(qcnt) struct mlx5e_qcounter_stats *qcnt = &priv->stats.qcnt; u32 out[MLX5_ST_SZ_DW(query_q_counter_out)] = {}; u32 in[MLX5_ST_SZ_DW(query_q_counter_in)] = {}; - int ret; + struct mlx5_core_dev *pos; + u32 rx_out_of_buffer = 0; + int ret, i; MLX5_SET(query_q_counter_in, in, opcode, MLX5_CMD_OP_QUERY_Q_COUNTER); - if (priv->q_counter) { - MLX5_SET(query_q_counter_in, in, counter_set_id, - priv->q_counter); - ret = mlx5_cmd_exec_inout(priv->mdev, query_q_counter, in, out); - if (!ret) - qcnt->rx_out_of_buffer = MLX5_GET(query_q_counter_out, - out, out_of_buffer); + mlx5_sd_for_each_dev(i, priv->mdev, pos) { + if (priv->q_counter[i]) { + MLX5_SET(query_q_counter_in, in, counter_set_id, + priv->q_counter[i]); + ret = mlx5_cmd_exec_inout(pos, query_q_counter, in, out); + if (!ret) + rx_out_of_buffer += MLX5_GET(query_q_counter_out, + out, out_of_buffer); + } } + qcnt->rx_out_of_buffer = rx_out_of_buffer; if (priv->drop_rq_q_counter) { MLX5_SET(query_q_counter_in, in, counter_set_id, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index 30932c9c9a8f..b8ceb972df9e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -766,7 +766,7 @@ static int mlx5e_hairpin_create_indirect_rqt(struct mlx5e_hairpin *hp) return err; mlx5e_rss_params_indir_init_uniform(&indir, hp->num_channels); - err = mlx5e_rqt_init_indir(&hp->indir_rqt, mdev, hp->pair->rqn, hp->num_channels, + err = mlx5e_rqt_init_indir(&hp->indir_rqt, mdev, hp->pair->rqn, NULL, hp->num_channels, mlx5e_rx_res_get_current_hash(priv->rx_res).hfunc, &indir); @@ -1169,7 +1169,7 @@ static int mlx5e_hairpin_flow_add(struct mlx5e_priv *priv, MLX5_CAP_GEN(priv->mdev, log_min_hairpin_wq_data_sz), MLX5_CAP_GEN(priv->mdev, log_max_hairpin_wq_data_sz)); - params.q_counter = priv->q_counter; + params.q_counter = priv->q_counter[0]; err = devl_param_driverinit_value_get( devlink, MLX5_DEVLINK_PARAM_ID_HAIRPIN_NUM_QUEUES, &val); if (err) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 3047d7015c52..3bf419d06d53 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1665,7 +1665,7 @@ int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 * void *hca_caps; int err; - if (!mlx5_core_is_ecpf(dev)) { + if (!mlx5_core_is_ecpf(dev) || mlx5_core_is_mgmt_pf(dev)) { *max_sfs = 0; return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c index 58845121954c..d77be1b4dd9c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c @@ -783,7 +783,7 @@ static int mlx5_rdma_setup_rn(struct ib_device *ibdev, u32 port_num, } /* This should only be called once per mdev */ - err = mlx5e_create_mdev_resources(mdev); + err = mlx5e_create_mdev_resources(mdev, false); if (err) goto destroy_ht; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h index ec32b686f586..d58032dd0df7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h @@ -10,6 +10,7 @@ enum mlx5_devcom_component { MLX5_DEVCOM_ESW_OFFLOADS, MLX5_DEVCOM_MPV, MLX5_DEVCOM_HCA_PORTS, + MLX5_DEVCOM_SD_GROUP, MLX5_DEVCOM_NUM_COMPONENTS, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h index 2b5826a785c4..37d5f445598c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h @@ -54,4 +54,16 @@ static inline struct net_device *mlx5_uplink_netdev_get(struct mlx5_core_dev *md { return mdev->mlx5e_res.uplink_netdev; } + +struct mlx5_sd; + +static inline struct mlx5_sd *mlx5_get_sd(struct mlx5_core_dev *dev) +{ + return dev->sd; +} + +static inline void mlx5_set_sd(struct mlx5_core_dev *dev, struct mlx5_sd *sd) +{ + dev->sd = sd; +} #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c new file mode 100644 index 000000000000..f68942277c62 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c @@ -0,0 +1,487 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include "lib/sd.h" +#include "mlx5_core.h" +#include "lib/mlx5.h" +#include "fs_cmd.h" +#include <linux/mlx5/vport.h> + +#define sd_info(__dev, format, ...) \ + dev_info((__dev)->device, "Socket-Direct: " format, ##__VA_ARGS__) +#define sd_warn(__dev, format, ...) \ + dev_warn((__dev)->device, "Socket-Direct: " format, ##__VA_ARGS__) + +struct mlx5_sd { + u32 group_id; + u8 host_buses; + struct mlx5_devcom_comp_dev *devcom; + bool primary; + union { + struct { /* primary */ + struct mlx5_core_dev *secondaries[MLX5_SD_MAX_GROUP_SZ - 1]; + struct mlx5_flow_table *tx_ft; + }; + struct { /* secondary */ + struct mlx5_core_dev *primary_dev; + u32 alias_obj_id; + }; + }; +}; + +static int mlx5_sd_get_host_buses(struct mlx5_core_dev *dev) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + + if (!sd) + return 1; + + return sd->host_buses; +} + +static struct mlx5_core_dev *mlx5_sd_get_primary(struct mlx5_core_dev *dev) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + + if (!sd) + return dev; + + return sd->primary ? dev : sd->primary_dev; +} + +struct mlx5_core_dev * +mlx5_sd_primary_get_peer(struct mlx5_core_dev *primary, int idx) +{ + struct mlx5_sd *sd; + + if (idx == 0) + return primary; + + if (idx >= mlx5_sd_get_host_buses(primary)) + return NULL; + + sd = mlx5_get_sd(primary); + return sd->secondaries[idx - 1]; +} + +int mlx5_sd_ch_ix_get_dev_ix(struct mlx5_core_dev *dev, int ch_ix) +{ + return ch_ix % mlx5_sd_get_host_buses(dev); +} + +int mlx5_sd_ch_ix_get_vec_ix(struct mlx5_core_dev *dev, int ch_ix) +{ + return ch_ix / mlx5_sd_get_host_buses(dev); +} + +struct mlx5_core_dev *mlx5_sd_ch_ix_get_dev(struct mlx5_core_dev *primary, int ch_ix) +{ + int mdev_idx = mlx5_sd_ch_ix_get_dev_ix(primary, ch_ix); + + return mlx5_sd_primary_get_peer(primary, mdev_idx); +} + +static bool ft_create_alias_supported(struct mlx5_core_dev *dev) +{ + u64 obj_allowed = MLX5_CAP_GEN_2_64(dev, allowed_object_for_other_vhca_access); + u32 obj_supp = MLX5_CAP_GEN_2(dev, cross_vhca_object_to_object_supported); + + if (!(obj_supp & + MLX5_CROSS_VHCA_OBJ_TO_OBJ_SUPPORTED_LOCAL_FLOW_TABLE_ROOT_TO_REMOTE_FLOW_TABLE)) + return false; + + if (!(obj_allowed & MLX5_ALLOWED_OBJ_FOR_OTHER_VHCA_ACCESS_FLOW_TABLE)) + return false; + + return true; +} + +static bool mlx5_sd_is_supported(struct mlx5_core_dev *dev, u8 host_buses) +{ + /* Feature is currently implemented for PFs only */ + if (!mlx5_core_is_pf(dev)) + return false; + + /* Honor the SW implementation limit */ + if (host_buses > MLX5_SD_MAX_GROUP_SZ) + return false; + + /* Disconnect secondaries from the network */ + if (!MLX5_CAP_GEN(dev, eswitch_manager)) + return false; + if (!MLX5_CAP_GEN(dev, silent_mode)) + return false; + + /* RX steering from primary to secondaries */ + if (!MLX5_CAP_GEN(dev, cross_vhca_rqt)) + return false; + if (host_buses > MLX5_CAP_GEN_2(dev, max_rqt_vhca_id)) + return false; + + /* TX steering from secondaries to primary */ + if (!ft_create_alias_supported(dev)) + return false; + if (!MLX5_CAP_FLOWTABLE_NIC_TX(dev, reset_root_to_default)) + return false; + + return true; +} + +static int mlx5_query_sd(struct mlx5_core_dev *dev, bool *sdm, + u8 *host_buses, u8 *sd_group) +{ + u32 out[MLX5_ST_SZ_DW(mpir_reg)]; + int err; + + err = mlx5_query_mpir_reg(dev, out); + if (err) + return err; + + err = mlx5_query_nic_vport_sd_group(dev, sd_group); + if (err) + return err; + + *sdm = MLX5_GET(mpir_reg, out, sdm); + *host_buses = MLX5_GET(mpir_reg, out, host_buses); + + return 0; +} + +static u32 mlx5_sd_group_id(struct mlx5_core_dev *dev, u8 sd_group) +{ + return (u32)((MLX5_CAP_GEN(dev, native_port_num) << 8) | sd_group); +} + +static int sd_init(struct mlx5_core_dev *dev) +{ + u8 host_buses, sd_group; + struct mlx5_sd *sd; + u32 group_id; + bool sdm; + int err; + + err = mlx5_query_sd(dev, &sdm, &host_buses, &sd_group); + if (err) + return err; + + if (!sdm) + return 0; + + if (!sd_group) + return 0; + + group_id = mlx5_sd_group_id(dev, sd_group); + + if (!mlx5_sd_is_supported(dev, host_buses)) { + sd_warn(dev, "can't support requested netdev combining for group id 0x%x), skipping\n", + group_id); + return 0; + } + + sd = kzalloc(sizeof(*sd), GFP_KERNEL); + if (!sd) + return -ENOMEM; + + sd->host_buses = host_buses; + sd->group_id = group_id; + + mlx5_set_sd(dev, sd); + + return 0; +} + +static void sd_cleanup(struct mlx5_core_dev *dev) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + + mlx5_set_sd(dev, NULL); + kfree(sd); +} + +static int sd_register(struct mlx5_core_dev *dev) +{ + struct mlx5_devcom_comp_dev *devcom, *pos; + struct mlx5_core_dev *peer, *primary; + struct mlx5_sd *sd, *primary_sd; + int err, i; + + sd = mlx5_get_sd(dev); + devcom = mlx5_devcom_register_component(dev->priv.devc, MLX5_DEVCOM_SD_GROUP, + sd->group_id, NULL, dev); + if (!devcom) + return -ENOMEM; + + sd->devcom = devcom; + + if (mlx5_devcom_comp_get_size(devcom) != sd->host_buses) + return 0; + + mlx5_devcom_comp_lock(devcom); + mlx5_devcom_comp_set_ready(devcom, true); + mlx5_devcom_comp_unlock(devcom); + + if (!mlx5_devcom_for_each_peer_begin(devcom)) { + err = -ENODEV; + goto err_devcom_unreg; + } + + primary = dev; + mlx5_devcom_for_each_peer_entry(devcom, peer, pos) + if (peer->pdev->bus->number < primary->pdev->bus->number) + primary = peer; + + primary_sd = mlx5_get_sd(primary); + primary_sd->primary = true; + i = 0; + /* loop the secondaries */ + mlx5_devcom_for_each_peer_entry(primary_sd->devcom, peer, pos) { + struct mlx5_sd *peer_sd = mlx5_get_sd(peer); + + primary_sd->secondaries[i++] = peer; + peer_sd->primary = false; + peer_sd->primary_dev = primary; + } + + mlx5_devcom_for_each_peer_end(devcom); + return 0; + +err_devcom_unreg: + mlx5_devcom_comp_lock(sd->devcom); + mlx5_devcom_comp_set_ready(sd->devcom, false); + mlx5_devcom_comp_unlock(sd->devcom); + mlx5_devcom_unregister_component(sd->devcom); + return err; +} + +static void sd_unregister(struct mlx5_core_dev *dev) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + + mlx5_devcom_comp_lock(sd->devcom); + mlx5_devcom_comp_set_ready(sd->devcom, false); + mlx5_devcom_comp_unlock(sd->devcom); + mlx5_devcom_unregister_component(sd->devcom); +} + +static int sd_cmd_set_primary(struct mlx5_core_dev *primary, u8 *alias_key) +{ + struct mlx5_cmd_allow_other_vhca_access_attr allow_attr = {}; + struct mlx5_sd *sd = mlx5_get_sd(primary); + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_namespace *nic_ns; + struct mlx5_flow_table *ft; + int err; + + nic_ns = mlx5_get_flow_namespace(primary, MLX5_FLOW_NAMESPACE_EGRESS); + if (!nic_ns) + return -EOPNOTSUPP; + + ft = mlx5_create_flow_table(nic_ns, &ft_attr); + if (IS_ERR(ft)) { + err = PTR_ERR(ft); + return err; + } + sd->tx_ft = ft; + memcpy(allow_attr.access_key, alias_key, ACCESS_KEY_LEN); + allow_attr.obj_type = MLX5_GENERAL_OBJECT_TYPES_FLOW_TABLE_ALIAS; + allow_attr.obj_id = (ft->type << FT_ID_FT_TYPE_OFFSET) | ft->id; + + err = mlx5_cmd_allow_other_vhca_access(primary, &allow_attr); + if (err) { + mlx5_core_err(primary, "Failed to allow other vhca access err=%d\n", + err); + mlx5_destroy_flow_table(ft); + return err; + } + + return 0; +} + +static void sd_cmd_unset_primary(struct mlx5_core_dev *primary) +{ + struct mlx5_sd *sd = mlx5_get_sd(primary); + + mlx5_destroy_flow_table(sd->tx_ft); +} + +static int sd_secondary_create_alias_ft(struct mlx5_core_dev *secondary, + struct mlx5_core_dev *primary, + struct mlx5_flow_table *ft, + u32 *obj_id, u8 *alias_key) +{ + u32 aliased_object_id = (ft->type << FT_ID_FT_TYPE_OFFSET) | ft->id; + u16 vhca_id_to_be_accessed = MLX5_CAP_GEN(primary, vhca_id); + struct mlx5_cmd_alias_obj_create_attr alias_attr = {}; + int ret; + + memcpy(alias_attr.access_key, alias_key, ACCESS_KEY_LEN); + alias_attr.obj_id = aliased_object_id; + alias_attr.obj_type = MLX5_GENERAL_OBJECT_TYPES_FLOW_TABLE_ALIAS; + alias_attr.vhca_id = vhca_id_to_be_accessed; + ret = mlx5_cmd_alias_obj_create(secondary, &alias_attr, obj_id); + if (ret) { + mlx5_core_err(secondary, "Failed to create alias object err=%d\n", + ret); + return ret; + } + + return 0; +} + +static void sd_secondary_destroy_alias_ft(struct mlx5_core_dev *secondary) +{ + struct mlx5_sd *sd = mlx5_get_sd(secondary); + + mlx5_cmd_alias_obj_destroy(secondary, sd->alias_obj_id, + MLX5_GENERAL_OBJECT_TYPES_FLOW_TABLE_ALIAS); +} + +static int sd_cmd_set_secondary(struct mlx5_core_dev *secondary, + struct mlx5_core_dev *primary, + u8 *alias_key) +{ + struct mlx5_sd *primary_sd = mlx5_get_sd(primary); + struct mlx5_sd *sd = mlx5_get_sd(secondary); + int err; + + err = mlx5_fs_cmd_set_l2table_entry_silent(secondary, 1); + if (err) + return err; + + err = sd_secondary_create_alias_ft(secondary, primary, primary_sd->tx_ft, + &sd->alias_obj_id, alias_key); + if (err) + goto err_unset_silent; + + err = mlx5_fs_cmd_set_tx_flow_table_root(secondary, sd->alias_obj_id, false); + if (err) + goto err_destroy_alias_ft; + + return 0; + +err_destroy_alias_ft: + sd_secondary_destroy_alias_ft(secondary); +err_unset_silent: + mlx5_fs_cmd_set_l2table_entry_silent(secondary, 0); + return err; +} + +static void sd_cmd_unset_secondary(struct mlx5_core_dev *secondary) +{ + mlx5_fs_cmd_set_tx_flow_table_root(secondary, 0, true); + sd_secondary_destroy_alias_ft(secondary); + mlx5_fs_cmd_set_l2table_entry_silent(secondary, 0); +} + +static void sd_print_group(struct mlx5_core_dev *primary) +{ + struct mlx5_sd *sd = mlx5_get_sd(primary); + struct mlx5_core_dev *pos; + int i; + + sd_info(primary, "group id %#x, primary %s, vhca %u\n", + sd->group_id, pci_name(primary->pdev), + MLX5_CAP_GEN(primary, vhca_id)); + mlx5_sd_for_each_secondary(i, primary, pos) + sd_info(primary, "group id %#x, secondary#%d %s, vhca %u\n", + sd->group_id, i - 1, pci_name(pos->pdev), + MLX5_CAP_GEN(pos, vhca_id)); +} + +int mlx5_sd_init(struct mlx5_core_dev *dev) +{ + struct mlx5_core_dev *primary, *pos, *to; + struct mlx5_sd *sd = mlx5_get_sd(dev); + u8 alias_key[ACCESS_KEY_LEN]; + int err, i; + + err = sd_init(dev); + if (err) + return err; + + sd = mlx5_get_sd(dev); + if (!sd) + return 0; + + err = sd_register(dev); + if (err) + goto err_sd_cleanup; + + if (!mlx5_devcom_comp_is_ready(sd->devcom)) + return 0; + + primary = mlx5_sd_get_primary(dev); + + for (i = 0; i < ACCESS_KEY_LEN; i++) + alias_key[i] = get_random_u8(); + + err = sd_cmd_set_primary(primary, alias_key); + if (err) + goto err_sd_unregister; + + mlx5_sd_for_each_secondary(i, primary, pos) { + err = sd_cmd_set_secondary(pos, primary, alias_key); + if (err) + goto err_unset_secondaries; + } + + sd_info(primary, "group id %#x, size %d, combined\n", + sd->group_id, mlx5_devcom_comp_get_size(sd->devcom)); + sd_print_group(primary); + + return 0; + +err_unset_secondaries: + to = pos; + mlx5_sd_for_each_secondary_to(i, primary, to, pos) + sd_cmd_unset_secondary(pos); + sd_cmd_unset_primary(primary); +err_sd_unregister: + sd_unregister(dev); +err_sd_cleanup: + sd_cleanup(dev); + return err; +} + +void mlx5_sd_cleanup(struct mlx5_core_dev *dev) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + struct mlx5_core_dev *primary, *pos; + int i; + + if (!sd) + return; + + if (!mlx5_devcom_comp_is_ready(sd->devcom)) + goto out; + + primary = mlx5_sd_get_primary(dev); + mlx5_sd_for_each_secondary(i, primary, pos) + sd_cmd_unset_secondary(pos); + sd_cmd_unset_primary(primary); + + sd_info(primary, "group id %#x, uncombined\n", sd->group_id); +out: + sd_unregister(dev); + sd_cleanup(dev); +} + +struct auxiliary_device *mlx5_sd_get_adev(struct mlx5_core_dev *dev, + struct auxiliary_device *adev, + int idx) +{ + struct mlx5_sd *sd = mlx5_get_sd(dev); + struct mlx5_core_dev *primary; + + if (!sd) + return adev; + + if (!mlx5_devcom_comp_is_ready(sd->devcom)) + return NULL; + + primary = mlx5_sd_get_primary(dev); + if (dev == primary) + return adev; + + return &primary->priv.adev[idx]->adev; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.h new file mode 100644 index 000000000000..137efaf9aabc --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_LIB_SD_H__ +#define __MLX5_LIB_SD_H__ + +#define MLX5_SD_MAX_GROUP_SZ 2 + +struct mlx5_sd; + +struct mlx5_core_dev *mlx5_sd_primary_get_peer(struct mlx5_core_dev *primary, int idx); +int mlx5_sd_ch_ix_get_dev_ix(struct mlx5_core_dev *dev, int ch_ix); +int mlx5_sd_ch_ix_get_vec_ix(struct mlx5_core_dev *dev, int ch_ix); +struct mlx5_core_dev *mlx5_sd_ch_ix_get_dev(struct mlx5_core_dev *primary, int ch_ix); +struct auxiliary_device *mlx5_sd_get_adev(struct mlx5_core_dev *dev, + struct auxiliary_device *adev, + int idx); + +int mlx5_sd_init(struct mlx5_core_dev *dev); +void mlx5_sd_cleanup(struct mlx5_core_dev *dev); + +#define mlx5_sd_for_each_dev_from_to(i, primary, ix_from, to, pos) \ + for (i = ix_from; \ + (pos = mlx5_sd_primary_get_peer(primary, i)) && pos != (to); i++) + +#define mlx5_sd_for_each_dev(i, primary, pos) \ + mlx5_sd_for_each_dev_from_to(i, primary, 0, NULL, pos) + +#define mlx5_sd_for_each_dev_to(i, primary, to, pos) \ + mlx5_sd_for_each_dev_from_to(i, primary, 0, to, pos) + +#define mlx5_sd_for_each_secondary(i, primary, pos) \ + mlx5_sd_for_each_dev_from_to(i, primary, 1, NULL, pos) + +#define mlx5_sd_for_each_secondary_to(i, primary, to, pos) \ + mlx5_sd_for_each_dev_from_to(i, primary, 1, to, pos) + +#endif /* __MLX5_LIB_SD_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c index 21753f327868..1005bb6935b6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c @@ -440,6 +440,27 @@ out: } EXPORT_SYMBOL_GPL(mlx5_query_nic_vport_system_image_guid); +int mlx5_query_nic_vport_sd_group(struct mlx5_core_dev *mdev, u8 *sd_group) +{ + int outlen = MLX5_ST_SZ_BYTES(query_nic_vport_context_out); + u32 *out; + int err; + + out = kvzalloc(outlen, GFP_KERNEL); + if (!out) + return -ENOMEM; + + err = mlx5_query_nic_vport_context(mdev, 0, out); + if (err) + goto out; + + *sd_group = MLX5_GET(query_nic_vport_context_out, out, + nic_vport_context.sd_group); +out: + kvfree(out); + return err; +} + int mlx5_query_nic_vport_node_guid(struct mlx5_core_dev *mdev, u64 *node_guid) { u32 *out; diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig index e60b557d59b9..be01450c20dc 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -134,14 +134,16 @@ config TI_K3_AM65_CPTS protocol, Ethernet Enhanced Scheduled Traffic Operations (CPTS_ESTFn) and PCIe Subsystem Precision Time Measurement (PTM). -config TI_AM65_CPSW_TAS - bool "Enable TAS offload in AM65 CPSW" +config TI_AM65_CPSW_QOS + bool "Enable QoS offload features in AM65 CPSW" depends on TI_K3_AM65_CPSW_NUSS && NET_SCH_TAPRIO && TI_K3_AM65_CPTS help - Say y here to support Time Aware Shaper(TAS) offload in AM65 CPSW. - AM65 CPSW hardware supports Enhanced Scheduled Traffic (EST) - defined in IEEE 802.1Q 2018. The EST scheduler runs on CPTS and the - TAS/EST schedule is updated in the Fetch RAM memory of the CPSW. + This option enables QoS offload features in AM65 CPSW like + Time Aware Shaper (TAS) / Enhanced Scheduled Traffic (EST), + MQPRIO qdisc offload and Frame-Preemption MAC Merge / Interspersing + Express Traffic (IET). + The EST scheduler runs on CPTS and the TAS/EST schedule is + updated in the Fetch RAM memory of the CPSW. config TI_KEYSTONE_NETCP tristate "TI Keystone NETCP Core Support" diff --git a/drivers/net/ethernet/ti/Makefile b/drivers/net/ethernet/ti/Makefile index 27de1d697134..d8590304f3df 100644 --- a/drivers/net/ethernet/ti/Makefile +++ b/drivers/net/ethernet/ti/Makefile @@ -26,7 +26,8 @@ keystone_netcp_ethss-y := netcp_ethss.o netcp_sgmii.o netcp_xgbepcsr.o cpsw_ale. obj-$(CONFIG_TI_K3_CPPI_DESC_POOL) += k3-cppi-desc-pool.o obj-$(CONFIG_TI_K3_AM65_CPSW_NUSS) += ti-am65-cpsw-nuss.o -ti-am65-cpsw-nuss-y := am65-cpsw-nuss.o cpsw_sl.o am65-cpsw-ethtool.o cpsw_ale.o am65-cpsw-qos.o +ti-am65-cpsw-nuss-y := am65-cpsw-nuss.o cpsw_sl.o am65-cpsw-ethtool.o cpsw_ale.o +ti-am65-cpsw-nuss-$(CONFIG_TI_AM65_CPSW_QOS) += am65-cpsw-qos.o ti-am65-cpsw-nuss-$(CONFIG_TI_K3_AM65_CPSW_SWITCHDEV) += am65-cpsw-switchdev.o obj-$(CONFIG_TI_K3_AM65_CPTS) += am65-cpts.o diff --git a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c index b9e1d568604b..35fceba01ea4 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c +++ b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c @@ -11,6 +11,7 @@ #include <linux/pm_runtime.h> #include "am65-cpsw-nuss.h" +#include "am65-cpsw-qos.h" #include "cpsw_ale.h" #include "am65-cpts.h" @@ -670,6 +671,9 @@ static void am65_cpsw_get_eth_mac_stats(struct net_device *ndev, stats = port->stat_base; + if (s->src != ETHTOOL_MAC_STATS_SRC_AGGREGATE) + return; + s->FramesTransmittedOK = readl_relaxed(&stats->tx_good_frames); s->SingleCollisionFrames = readl_relaxed(&stats->tx_single_coll_frames); s->MultipleCollisionFrames = readl_relaxed(&stats->tx_mult_coll_frames); @@ -740,6 +744,240 @@ static int am65_cpsw_set_ethtool_priv_flags(struct net_device *ndev, u32 flags) return 0; } +static void am65_cpsw_port_iet_rx_enable(struct am65_cpsw_port *port, bool enable) +{ + u32 val; + + val = readl(port->port_base + AM65_CPSW_PN_REG_CTL); + if (enable) + val |= AM65_CPSW_PN_CTL_IET_PORT_EN; + else + val &= ~AM65_CPSW_PN_CTL_IET_PORT_EN; + + writel(val, port->port_base + AM65_CPSW_PN_REG_CTL); + am65_cpsw_iet_common_enable(port->common); +} + +static void am65_cpsw_port_iet_tx_enable(struct am65_cpsw_port *port, bool enable) +{ + u32 val; + + val = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + if (enable) + val |= AM65_CPSW_PN_IET_MAC_PENABLE; + else + val &= ~AM65_CPSW_PN_IET_MAC_PENABLE; + + writel(val, port->port_base + AM65_CPSW_PN_REG_IET_CTRL); +} + +static int am65_cpsw_get_mm(struct net_device *ndev, struct ethtool_mm_state *state) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + struct am65_cpsw_ndev_priv *priv = netdev_priv(ndev); + u32 port_ctrl, iet_ctrl, iet_status; + u32 add_frag_size; + + if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_QOS)) + return -EOPNOTSUPP; + + mutex_lock(&priv->mm_lock); + + iet_ctrl = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + port_ctrl = readl(port->port_base + AM65_CPSW_PN_REG_CTL); + + state->tx_enabled = !!(iet_ctrl & AM65_CPSW_PN_IET_MAC_PENABLE); + state->pmac_enabled = !!(port_ctrl & AM65_CPSW_PN_CTL_IET_PORT_EN); + + iet_status = readl(port->port_base + AM65_CPSW_PN_REG_IET_STATUS); + + if (iet_ctrl & AM65_CPSW_PN_IET_MAC_DISABLEVERIFY) + state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED; + else if (iet_status & AM65_CPSW_PN_MAC_VERIFIED) + state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED; + else if (iet_status & AM65_CPSW_PN_MAC_VERIFY_FAIL) + state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED; + else + state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN; + + add_frag_size = AM65_CPSW_PN_IET_MAC_GET_ADDFRAGSIZE(iet_ctrl); + state->tx_min_frag_size = ethtool_mm_frag_size_add_to_min(add_frag_size); + + /* Errata i2208: RX min fragment size cannot be less than 124 */ + state->rx_min_frag_size = 124; + + /* FPE active if common tx_enabled and verification success or disabled (forced) */ + state->tx_active = state->tx_enabled && + (state->verify_status == ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED || + state->verify_status == ETHTOOL_MM_VERIFY_STATUS_DISABLED); + state->verify_enabled = !(iet_ctrl & AM65_CPSW_PN_IET_MAC_DISABLEVERIFY); + + state->verify_time = port->qos.iet.verify_time_ms; + + /* 802.3-2018 clause 30.14.1.6, says that the aMACMergeVerifyTime + * variable has a range between 1 and 128 ms inclusive. Limit to that. + */ + state->max_verify_time = 128; + + mutex_unlock(&priv->mm_lock); + + return 0; +} + +static int am65_cpsw_set_mm(struct net_device *ndev, struct ethtool_mm_cfg *cfg, + struct netlink_ext_ack *extack) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + struct am65_cpsw_ndev_priv *priv = netdev_priv(ndev); + struct am65_cpsw_iet *iet = &port->qos.iet; + u32 val, add_frag_size; + int err; + + if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_QOS)) + return -EOPNOTSUPP; + + err = ethtool_mm_frag_size_min_to_add(cfg->tx_min_frag_size, &add_frag_size, extack); + if (err) + return err; + + mutex_lock(&priv->mm_lock); + + if (cfg->pmac_enabled) { + /* change TX & RX FIFO MAX_BLKS as per TRM recommendation */ + if (!iet->original_max_blks) + iet->original_max_blks = readl(port->port_base + AM65_CPSW_PN_REG_MAX_BLKS); + + writel(AM65_CPSW_PN_TX_RX_MAX_BLKS_IET, + port->port_base + AM65_CPSW_PN_REG_MAX_BLKS); + } else if (iet->original_max_blks) { + /* restore RX & TX FIFO MAX_BLKS */ + writel(iet->original_max_blks, + port->port_base + AM65_CPSW_PN_REG_MAX_BLKS); + } + + am65_cpsw_port_iet_rx_enable(port, cfg->pmac_enabled); + am65_cpsw_port_iet_tx_enable(port, cfg->tx_enabled); + + val = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + if (cfg->verify_enabled) { + val &= ~AM65_CPSW_PN_IET_MAC_DISABLEVERIFY; + /* Reset Verify state machine. Verification won't start here. + * Verification will be done once link-up. + */ + val |= AM65_CPSW_PN_IET_MAC_LINKFAIL; + } else { + val |= AM65_CPSW_PN_IET_MAC_DISABLEVERIFY; + /* Clear LINKFAIL to allow verify/response packets */ + val &= ~AM65_CPSW_PN_IET_MAC_LINKFAIL; + } + + val &= ~AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_MASK; + val |= AM65_CPSW_PN_IET_MAC_SET_ADDFRAGSIZE(add_frag_size); + writel(val, port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + + /* verify_timeout_count can only be set at valid link */ + port->qos.iet.verify_time_ms = cfg->verify_time; + + /* enable/disable preemption based on link status */ + am65_cpsw_iet_commit_preemptible_tcs(port); + + mutex_unlock(&priv->mm_lock); + + return 0; +} + +static void am65_cpsw_get_mm_stats(struct net_device *ndev, + struct ethtool_mm_stats *s) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + void __iomem *base = port->stat_base; + + s->MACMergeFrameAssOkCount = readl(base + AM65_CPSW_STATN_IET_RX_ASSEMBLY_OK); + s->MACMergeFrameAssErrorCount = readl(base + AM65_CPSW_STATN_IET_RX_ASSEMBLY_ERROR); + s->MACMergeFrameSmdErrorCount = readl(base + AM65_CPSW_STATN_IET_RX_SMD_ERROR); + /* CPSW Functional Spec states: + * "The IET stat aMACMergeFragCountRx is derived by adding the + * Receive Assembly Error count to this value. i.e. AM65_CPSW_STATN_IET_RX_FRAG" + */ + s->MACMergeFragCountRx = readl(base + AM65_CPSW_STATN_IET_RX_FRAG) + s->MACMergeFrameAssErrorCount; + s->MACMergeFragCountTx = readl(base + AM65_CPSW_STATN_IET_TX_FRAG); + s->MACMergeHoldCount = readl(base + AM65_CPSW_STATN_IET_TX_HOLD); +} + +static int am65_cpsw_get_coalesce(struct net_device *ndev, struct ethtool_coalesce *coal, + struct kernel_ethtool_coalesce *kernel_coal, + struct netlink_ext_ack *extack) +{ + struct am65_cpsw_common *common = am65_ndev_to_common(ndev); + struct am65_cpsw_tx_chn *tx_chn; + + tx_chn = &common->tx_chns[0]; + + coal->rx_coalesce_usecs = common->rx_pace_timeout / 1000; + coal->tx_coalesce_usecs = tx_chn->tx_pace_timeout / 1000; + + return 0; +} + +static int am65_cpsw_get_per_queue_coalesce(struct net_device *ndev, u32 queue, + struct ethtool_coalesce *coal) +{ + struct am65_cpsw_common *common = am65_ndev_to_common(ndev); + struct am65_cpsw_tx_chn *tx_chn; + + if (queue >= AM65_CPSW_MAX_TX_QUEUES) + return -EINVAL; + + tx_chn = &common->tx_chns[queue]; + + coal->tx_coalesce_usecs = tx_chn->tx_pace_timeout / 1000; + + return 0; +} + +static int am65_cpsw_set_coalesce(struct net_device *ndev, struct ethtool_coalesce *coal, + struct kernel_ethtool_coalesce *kernel_coal, + struct netlink_ext_ack *extack) +{ + struct am65_cpsw_common *common = am65_ndev_to_common(ndev); + struct am65_cpsw_tx_chn *tx_chn; + + tx_chn = &common->tx_chns[0]; + + if (coal->rx_coalesce_usecs && coal->rx_coalesce_usecs < 20) + return -EINVAL; + + if (coal->tx_coalesce_usecs && coal->tx_coalesce_usecs < 20) + return -EINVAL; + + common->rx_pace_timeout = coal->rx_coalesce_usecs * 1000; + tx_chn->tx_pace_timeout = coal->tx_coalesce_usecs * 1000; + + return 0; +} + +static int am65_cpsw_set_per_queue_coalesce(struct net_device *ndev, u32 queue, + struct ethtool_coalesce *coal) +{ + struct am65_cpsw_common *common = am65_ndev_to_common(ndev); + struct am65_cpsw_tx_chn *tx_chn; + + if (queue >= AM65_CPSW_MAX_TX_QUEUES) + return -EINVAL; + + tx_chn = &common->tx_chns[queue]; + + if (coal->tx_coalesce_usecs && coal->tx_coalesce_usecs < 20) { + dev_info(common->dev, "defaulting to min value of 20us for tx-usecs for tx-%u\n", + queue); + coal->tx_coalesce_usecs = 20; + } + + tx_chn->tx_pace_timeout = coal->tx_coalesce_usecs * 1000; + + return 0; +} + const struct ethtool_ops am65_cpsw_ethtool_ops_slave = { .begin = am65_cpsw_ethtool_op_begin, .complete = am65_cpsw_ethtool_op_complete, @@ -758,6 +996,11 @@ const struct ethtool_ops am65_cpsw_ethtool_ops_slave = { .get_ts_info = am65_cpsw_get_ethtool_ts_info, .get_priv_flags = am65_cpsw_get_ethtool_priv_flags, .set_priv_flags = am65_cpsw_set_ethtool_priv_flags, + .supported_coalesce_params = ETHTOOL_COALESCE_USECS, + .get_coalesce = am65_cpsw_get_coalesce, + .set_coalesce = am65_cpsw_set_coalesce, + .get_per_queue_coalesce = am65_cpsw_get_per_queue_coalesce, + .set_per_queue_coalesce = am65_cpsw_set_per_queue_coalesce, .get_link = ethtool_op_get_link, .get_link_ksettings = am65_cpsw_get_link_ksettings, @@ -769,4 +1012,7 @@ const struct ethtool_ops am65_cpsw_ethtool_ops_slave = { .get_eee = am65_cpsw_get_eee, .set_eee = am65_cpsw_set_eee, .nway_reset = am65_cpsw_nway_reset, + .get_mm = am65_cpsw_get_mm, + .set_mm = am65_cpsw_set_mm, + .get_mm_stats = am65_cpsw_get_mm_stats, }; diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index 7651f90f51f2..faa0561e988e 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -596,8 +596,10 @@ static int am65_cpsw_nuss_common_stop(struct am65_cpsw_common *common) msecs_to_jiffies(1000)); if (!i) dev_err(common->dev, "tx timeout\n"); - for (i = 0; i < common->tx_ch_num; i++) + for (i = 0; i < common->tx_ch_num; i++) { napi_disable(&common->tx_chns[i].napi_tx); + hrtimer_cancel(&common->tx_chns[i].tx_hrtimer); + } for (i = 0; i < common->tx_ch_num; i++) { k3_udma_glue_reset_tx_chn(common->tx_chns[i].tx_chn, @@ -616,6 +618,7 @@ static int am65_cpsw_nuss_common_stop(struct am65_cpsw_common *common) } napi_disable(&common->napi_rx); + hrtimer_cancel(&common->rx_hrtimer); for (i = 0; i < AM65_CPSW_MAX_RX_FLOWS; i++) k3_udma_glue_reset_rx_chn(common->rx_chns.rx_chn, i, @@ -885,6 +888,15 @@ static int am65_cpsw_nuss_rx_packets(struct am65_cpsw_common *common, return ret; } +static enum hrtimer_restart am65_cpsw_nuss_rx_timer_callback(struct hrtimer *timer) +{ + struct am65_cpsw_common *common = + container_of(timer, struct am65_cpsw_common, rx_hrtimer); + + enable_irq(common->rx_chns.irq); + return HRTIMER_NORESTART; +} + static int am65_cpsw_nuss_rx_poll(struct napi_struct *napi_rx, int budget) { struct am65_cpsw_common *common = am65_cpsw_napi_to_common(napi_rx); @@ -912,7 +924,13 @@ static int am65_cpsw_nuss_rx_poll(struct napi_struct *napi_rx, int budget) if (num_rx < budget && napi_complete_done(napi_rx, num_rx)) { if (common->rx_irq_disabled) { common->rx_irq_disabled = false; - enable_irq(common->rx_chns.irq); + if (unlikely(common->rx_pace_timeout)) { + hrtimer_start(&common->rx_hrtimer, + ns_to_ktime(common->rx_pace_timeout), + HRTIMER_MODE_REL_PINNED); + } else { + enable_irq(common->rx_chns.irq); + } } } @@ -968,7 +986,7 @@ static void am65_cpsw_nuss_tx_wake(struct am65_cpsw_tx_chn *tx_chn, struct net_d } static int am65_cpsw_nuss_tx_compl_packets(struct am65_cpsw_common *common, - int chn, unsigned int budget) + int chn, unsigned int budget, bool *tdown) { struct device *dev = common->dev; struct am65_cpsw_tx_chn *tx_chn; @@ -991,6 +1009,7 @@ static int am65_cpsw_nuss_tx_compl_packets(struct am65_cpsw_common *common, if (cppi5_desc_is_tdcm(desc_dma)) { if (atomic_dec_and_test(&common->tdown_cnt)) complete(&common->tdown_complete); + *tdown = true; break; } @@ -1013,7 +1032,7 @@ static int am65_cpsw_nuss_tx_compl_packets(struct am65_cpsw_common *common, } static int am65_cpsw_nuss_tx_compl_packets_2g(struct am65_cpsw_common *common, - int chn, unsigned int budget) + int chn, unsigned int budget, bool *tdown) { struct device *dev = common->dev; struct am65_cpsw_tx_chn *tx_chn; @@ -1034,6 +1053,7 @@ static int am65_cpsw_nuss_tx_compl_packets_2g(struct am65_cpsw_common *common, if (cppi5_desc_is_tdcm(desc_dma)) { if (atomic_dec_and_test(&common->tdown_cnt)) complete(&common->tdown_complete); + *tdown = true; break; } @@ -1059,21 +1079,40 @@ static int am65_cpsw_nuss_tx_compl_packets_2g(struct am65_cpsw_common *common, return num_tx; } +static enum hrtimer_restart am65_cpsw_nuss_tx_timer_callback(struct hrtimer *timer) +{ + struct am65_cpsw_tx_chn *tx_chns = + container_of(timer, struct am65_cpsw_tx_chn, tx_hrtimer); + + enable_irq(tx_chns->irq); + return HRTIMER_NORESTART; +} + static int am65_cpsw_nuss_tx_poll(struct napi_struct *napi_tx, int budget) { struct am65_cpsw_tx_chn *tx_chn = am65_cpsw_napi_to_tx_chn(napi_tx); + bool tdown = false; int num_tx; if (AM65_CPSW_IS_CPSW2G(tx_chn->common)) - num_tx = am65_cpsw_nuss_tx_compl_packets_2g(tx_chn->common, tx_chn->id, budget); + num_tx = am65_cpsw_nuss_tx_compl_packets_2g(tx_chn->common, tx_chn->id, + budget, &tdown); else - num_tx = am65_cpsw_nuss_tx_compl_packets(tx_chn->common, tx_chn->id, budget); + num_tx = am65_cpsw_nuss_tx_compl_packets(tx_chn->common, + tx_chn->id, budget, &tdown); if (num_tx >= budget) return budget; - if (napi_complete_done(napi_tx, num_tx)) - enable_irq(tx_chn->irq); + if (napi_complete_done(napi_tx, num_tx)) { + if (unlikely(tx_chn->tx_pace_timeout && !tdown)) { + hrtimer_start(&tx_chn->tx_hrtimer, + ns_to_ktime(tx_chn->tx_pace_timeout), + HRTIMER_MODE_REL_PINNED); + } else { + enable_irq(tx_chn->irq); + } + } return 0; } @@ -1705,6 +1744,8 @@ static int am65_cpsw_nuss_ndev_add_tx_napi(struct am65_cpsw_common *common) netif_napi_add_tx(common->dma_ndev, &tx_chn->napi_tx, am65_cpsw_nuss_tx_poll); + hrtimer_init(&tx_chn->tx_hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); + tx_chn->tx_hrtimer.function = &am65_cpsw_nuss_tx_timer_callback; ret = devm_request_irq(dev, tx_chn->irq, am65_cpsw_nuss_tx_irq, @@ -1930,6 +1971,8 @@ static int am65_cpsw_nuss_init_rx_chns(struct am65_cpsw_common *common) netif_napi_add(common->dma_ndev, &common->napi_rx, am65_cpsw_nuss_rx_poll); + hrtimer_init(&common->rx_hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); + common->rx_hrtimer.function = &am65_cpsw_nuss_rx_timer_callback; ret = devm_request_irq(dev, rx_chn->irq, am65_cpsw_nuss_rx_irq, @@ -2127,6 +2170,9 @@ static int am65_cpsw_nuss_init_slave_ports(struct am65_cpsw_common *common) dev_err(dev, "Use random MAC address\n"); } } + + /* Reset all Queue priorities to 0 */ + writel(0, port->port_base + AM65_CPSW_PN_REG_TX_PRI_MAP); } of_node_put(node); @@ -2191,6 +2237,8 @@ am65_cpsw_nuss_init_port_ndev(struct am65_cpsw_common *common, u32 port_idx) ndev_priv = netdev_priv(port->ndev); ndev_priv->port = port; ndev_priv->msg_enable = AM65_CPSW_DEBUG; + mutex_init(&ndev_priv->mm_lock); + port->qos.link_speed = SPEED_UNKNOWN; SET_NETDEV_DEV(port->ndev, dev); eth_hw_addr_set(port->ndev, port->slave.mac_addr); diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.h b/drivers/net/ethernet/ti/am65-cpsw-nuss.h index f3dad2ab9828..7da0492dc091 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.h +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.h @@ -75,6 +75,8 @@ struct am65_cpsw_tx_chn { struct k3_cppi_desc_pool *desc_pool; struct k3_udma_glue_tx_channel *tx_chn; spinlock_t lock; /* protect TX rings in multi-port mode */ + struct hrtimer tx_hrtimer; + unsigned long tx_pace_timeout; int irq; u32 id; u32 descs_num; @@ -138,6 +140,8 @@ struct am65_cpsw_common { struct napi_struct napi_rx; bool rx_irq_disabled; + struct hrtimer rx_hrtimer; + unsigned long rx_pace_timeout; u32 nuss_ver; u32 cpsw_ver; @@ -145,6 +149,7 @@ struct am65_cpsw_common { bool pf_p0_rx_ptype_rrobin; struct am65_cpts *cpts; int est_enabled; + bool iet_enabled; bool is_emac_mode; u16 br_members; @@ -170,6 +175,10 @@ struct am65_cpsw_ndev_priv { struct am65_cpsw_port *port; struct am65_cpsw_ndev_stats __percpu *stats; bool offload_fwd_mark; + /* Serialize access to MAC Merge state between ethtool requests + * and link state updates + */ + struct mutex mm_lock; }; #define am65_ndev_to_priv(ndev) \ diff --git a/drivers/net/ethernet/ti/am65-cpsw-qos.c b/drivers/net/ethernet/ti/am65-cpsw-qos.c index 9ac2ff05d501..816e73a3d6e4 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-qos.c +++ b/drivers/net/ethernet/ti/am65-cpsw-qos.c @@ -4,10 +4,13 @@ * * quality of service module includes: * Enhanced Scheduler Traffic (EST - P802.1Qbv/D2.2) + * Interspersed Express Traffic (IET - P802.3br/D2.0) */ #include <linux/pm_runtime.h> +#include <linux/math.h> #include <linux/time.h> +#include <linux/units.h> #include <net/pkt_cls.h> #include "am65-cpsw-nuss.h" @@ -15,40 +18,7 @@ #include "am65-cpts.h" #include "cpsw_ale.h" -#define AM65_CPSW_REG_CTL 0x004 -#define AM65_CPSW_PN_REG_CTL 0x004 -#define AM65_CPSW_PN_REG_FIFO_STATUS 0x050 -#define AM65_CPSW_PN_REG_EST_CTL 0x060 -#define AM65_CPSW_PN_REG_PRI_CIR(pri) (0x140 + 4 * (pri)) - -/* AM65_CPSW_REG_CTL register fields */ -#define AM65_CPSW_CTL_EST_EN BIT(18) - -/* AM65_CPSW_PN_REG_CTL register fields */ -#define AM65_CPSW_PN_CTL_EST_PORT_EN BIT(17) - -/* AM65_CPSW_PN_REG_EST_CTL register fields */ -#define AM65_CPSW_PN_EST_ONEBUF BIT(0) -#define AM65_CPSW_PN_EST_BUFSEL BIT(1) -#define AM65_CPSW_PN_EST_TS_EN BIT(2) -#define AM65_CPSW_PN_EST_TS_FIRST BIT(3) -#define AM65_CPSW_PN_EST_ONEPRI BIT(4) -#define AM65_CPSW_PN_EST_TS_PRI_MSK GENMASK(7, 5) - -/* AM65_CPSW_PN_REG_FIFO_STATUS register fields */ -#define AM65_CPSW_PN_FST_TX_PRI_ACTIVE_MSK GENMASK(7, 0) -#define AM65_CPSW_PN_FST_TX_E_MAC_ALLOW_MSK GENMASK(15, 8) -#define AM65_CPSW_PN_FST_EST_CNT_ERR BIT(16) -#define AM65_CPSW_PN_FST_EST_ADD_ERR BIT(17) -#define AM65_CPSW_PN_FST_EST_BUFACT BIT(18) - -/* EST FETCH COMMAND RAM */ -#define AM65_CPSW_FETCH_RAM_CMD_NUM 0x80 -#define AM65_CPSW_FETCH_CNT_MSK GENMASK(21, 8) -#define AM65_CPSW_FETCH_CNT_MAX (AM65_CPSW_FETCH_CNT_MSK >> 8) -#define AM65_CPSW_FETCH_CNT_OFFSET 8 -#define AM65_CPSW_FETCH_ALLOW_MSK GENMASK(7, 0) -#define AM65_CPSW_FETCH_ALLOW_MAX AM65_CPSW_FETCH_ALLOW_MSK +#define TO_MBPS(x) DIV_ROUND_UP((x), BYTES_PER_MBIT) enum timer_act { TACT_PROG, /* need program timer */ @@ -56,6 +26,412 @@ enum timer_act { TACT_SKIP_PROG, /* just buffer can be updated */ }; +static void am65_cpsw_iet_change_preemptible_tcs(struct am65_cpsw_port *port, u8 preemptible_tcs); + +static u32 +am65_cpsw_qos_tx_rate_calc(u32 rate_mbps, unsigned long bus_freq) +{ + u32 ir; + + bus_freq /= 1000000; + ir = DIV_ROUND_UP(((u64)rate_mbps * 32768), bus_freq); + return ir; +} + +static void am65_cpsw_tx_pn_shaper_reset(struct am65_cpsw_port *port) +{ + int prio; + + for (prio = 0; prio < AM65_CPSW_PN_FIFO_PRIO_NUM; prio++) { + writel(0, port->port_base + AM65_CPSW_PN_REG_PRI_CIR(prio)); + writel(0, port->port_base + AM65_CPSW_PN_REG_PRI_EIR(prio)); + } +} + +static void am65_cpsw_tx_pn_shaper_apply(struct am65_cpsw_port *port) +{ + struct am65_cpsw_mqprio *p_mqprio = &port->qos.mqprio; + struct am65_cpsw_common *common = port->common; + struct tc_mqprio_qopt_offload *mqprio; + bool enable, shaper_susp = false; + u32 rate_mbps; + int tc, prio; + + mqprio = &p_mqprio->mqprio_hw; + /* takes care of no link case as well */ + if (p_mqprio->max_rate_total > port->qos.link_speed) + shaper_susp = true; + + am65_cpsw_tx_pn_shaper_reset(port); + + enable = p_mqprio->shaper_en && !shaper_susp; + if (!enable) + return; + + /* Rate limit is specified per Traffic Class but + * for CPSW, rate limit can be applied per priority + * at port FIFO. + * + * We have assigned the same priority (TCn) to all queues + * of a Traffic Class so they share the same shaper + * bandwidth. + */ + for (tc = 0; tc < mqprio->qopt.num_tc; tc++) { + prio = tc; + + rate_mbps = TO_MBPS(mqprio->min_rate[tc]); + rate_mbps = am65_cpsw_qos_tx_rate_calc(rate_mbps, + common->bus_freq); + writel(rate_mbps, + port->port_base + AM65_CPSW_PN_REG_PRI_CIR(prio)); + + rate_mbps = 0; + + if (mqprio->max_rate[tc]) { + rate_mbps = mqprio->max_rate[tc] - mqprio->min_rate[tc]; + rate_mbps = TO_MBPS(rate_mbps); + rate_mbps = am65_cpsw_qos_tx_rate_calc(rate_mbps, + common->bus_freq); + } + + writel(rate_mbps, + port->port_base + AM65_CPSW_PN_REG_PRI_EIR(prio)); + } +} + +static int am65_cpsw_mqprio_verify_shaper(struct am65_cpsw_port *port, + struct tc_mqprio_qopt_offload *mqprio) +{ + struct am65_cpsw_mqprio *p_mqprio = &port->qos.mqprio; + struct netlink_ext_ack *extack = mqprio->extack; + u64 min_rate_total = 0, max_rate_total = 0; + u32 min_rate_msk = 0, max_rate_msk = 0; + bool has_min_rate, has_max_rate; + int num_tc, i; + + if (!(mqprio->flags & TC_MQPRIO_F_SHAPER)) + return 0; + + if (mqprio->shaper != TC_MQPRIO_SHAPER_BW_RATE) + return 0; + + has_min_rate = !!(mqprio->flags & TC_MQPRIO_F_MIN_RATE); + has_max_rate = !!(mqprio->flags & TC_MQPRIO_F_MAX_RATE); + + if (!has_min_rate && has_max_rate) { + NL_SET_ERR_MSG_MOD(extack, "min_rate is required with max_rate"); + return -EOPNOTSUPP; + } + + if (!has_min_rate) + return 0; + + num_tc = mqprio->qopt.num_tc; + + for (i = num_tc - 1; i >= 0; i--) { + u32 ch_msk; + + if (mqprio->min_rate[i]) + min_rate_msk |= BIT(i); + min_rate_total += mqprio->min_rate[i]; + + if (has_max_rate) { + if (mqprio->max_rate[i]) + max_rate_msk |= BIT(i); + max_rate_total += mqprio->max_rate[i]; + + if (!mqprio->min_rate[i] && mqprio->max_rate[i]) { + NL_SET_ERR_MSG_FMT_MOD(extack, + "TX tc%d rate max>0 but min=0", + i); + return -EINVAL; + } + + if (mqprio->max_rate[i] && + mqprio->max_rate[i] < mqprio->min_rate[i]) { + NL_SET_ERR_MSG_FMT_MOD(extack, + "TX tc%d rate min(%llu)>max(%llu)", + i, mqprio->min_rate[i], + mqprio->max_rate[i]); + return -EINVAL; + } + } + + ch_msk = GENMASK(num_tc - 1, i); + if ((min_rate_msk & BIT(i)) && (min_rate_msk ^ ch_msk)) { + NL_SET_ERR_MSG_FMT_MOD(extack, + "Min rate must be set sequentially hi->lo tx_rate_msk%x", + min_rate_msk); + return -EINVAL; + } + + if ((max_rate_msk & BIT(i)) && (max_rate_msk ^ ch_msk)) { + NL_SET_ERR_MSG_FMT_MOD(extack, + "Max rate must be set sequentially hi->lo tx_rate_msk%x", + max_rate_msk); + return -EINVAL; + } + } + + min_rate_total = TO_MBPS(min_rate_total); + max_rate_total = TO_MBPS(max_rate_total); + + p_mqprio->shaper_en = true; + p_mqprio->max_rate_total = max_t(u64, min_rate_total, max_rate_total); + + return 0; +} + +static void am65_cpsw_reset_tc_mqprio(struct net_device *ndev) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + struct am65_cpsw_mqprio *p_mqprio = &port->qos.mqprio; + + p_mqprio->shaper_en = false; + p_mqprio->max_rate_total = 0; + + am65_cpsw_tx_pn_shaper_reset(port); + netdev_reset_tc(ndev); + + /* Reset all Queue priorities to 0 */ + writel(0, port->port_base + AM65_CPSW_PN_REG_TX_PRI_MAP); + + am65_cpsw_iet_change_preemptible_tcs(port, 0); +} + +static int am65_cpsw_setup_mqprio(struct net_device *ndev, void *type_data) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + struct am65_cpsw_mqprio *p_mqprio = &port->qos.mqprio; + struct tc_mqprio_qopt_offload *mqprio = type_data; + struct am65_cpsw_common *common = port->common; + struct tc_mqprio_qopt *qopt = &mqprio->qopt; + int i, tc, offset, count, prio, ret; + u8 num_tc = qopt->num_tc; + u32 tx_prio_map = 0; + + memcpy(&p_mqprio->mqprio_hw, mqprio, sizeof(*mqprio)); + + ret = pm_runtime_get_sync(common->dev); + if (ret < 0) { + pm_runtime_put_noidle(common->dev); + return ret; + } + + if (!num_tc) { + am65_cpsw_reset_tc_mqprio(ndev); + ret = 0; + goto exit_put; + } + + ret = am65_cpsw_mqprio_verify_shaper(port, mqprio); + if (ret) + goto exit_put; + + netdev_set_num_tc(ndev, num_tc); + + /* Multiple Linux priorities can map to a Traffic Class + * A Traffic Class can have multiple contiguous Queues, + * Queues get mapped to Channels (thread_id), + * if not VLAN tagged, thread_id is used as packet_priority + * if VLAN tagged. VLAN priority is used as packet_priority + * packet_priority gets mapped to header_priority in p0_rx_pri_map, + * header_priority gets mapped to switch_priority in pn_tx_pri_map. + * As p0_rx_pri_map is left at defaults (0x76543210), we can + * assume that Queue_n gets mapped to header_priority_n. We can then + * set the switch priority in pn_tx_pri_map. + */ + + for (tc = 0; tc < num_tc; tc++) { + prio = tc; + + /* For simplicity we assign the same priority (TCn) to + * all queues of a Traffic Class. + */ + for (i = qopt->offset[tc]; i < qopt->offset[tc] + qopt->count[tc]; i++) + tx_prio_map |= prio << (4 * i); + + count = qopt->count[tc]; + offset = qopt->offset[tc]; + netdev_set_tc_queue(ndev, tc, count, offset); + } + + writel(tx_prio_map, port->port_base + AM65_CPSW_PN_REG_TX_PRI_MAP); + + am65_cpsw_tx_pn_shaper_apply(port); + am65_cpsw_iet_change_preemptible_tcs(port, mqprio->preemptible_tcs); + +exit_put: + pm_runtime_put(common->dev); + + return ret; +} + +static int am65_cpsw_iet_set_verify_timeout_count(struct am65_cpsw_port *port) +{ + int verify_time_ms = port->qos.iet.verify_time_ms; + u32 val; + + /* The number of wireside clocks contained in the verify + * timeout counter. The default is 0x1312d0 + * (10ms at 125Mhz in 1G mode). + */ + val = 125 * HZ_PER_MHZ; /* assuming 125MHz wireside clock */ + + val /= MILLIHZ_PER_HZ; /* count per ms timeout */ + val *= verify_time_ms; /* count for timeout ms */ + + if (val > AM65_CPSW_PN_MAC_VERIFY_CNT_MASK) + return -EINVAL; + + writel(val, port->port_base + AM65_CPSW_PN_REG_IET_VERIFY); + + return 0; +} + +static int am65_cpsw_iet_verify_wait(struct am65_cpsw_port *port) +{ + u32 ctrl, status; + int try; + + try = 20; + do { + /* Reset the verify state machine by writing 1 + * to LINKFAIL + */ + ctrl = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + ctrl |= AM65_CPSW_PN_IET_MAC_LINKFAIL; + writel(ctrl, port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + + /* Clear MAC_LINKFAIL bit to start Verify. */ + ctrl = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + ctrl &= ~AM65_CPSW_PN_IET_MAC_LINKFAIL; + writel(ctrl, port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + + msleep(port->qos.iet.verify_time_ms); + + status = readl(port->port_base + AM65_CPSW_PN_REG_IET_STATUS); + if (status & AM65_CPSW_PN_MAC_VERIFIED) + return 0; + + if (status & AM65_CPSW_PN_MAC_VERIFY_FAIL) { + netdev_dbg(port->ndev, + "MAC Merge verify failed, trying again\n"); + continue; + } + + if (status & AM65_CPSW_PN_MAC_RESPOND_ERR) { + netdev_dbg(port->ndev, "MAC Merge respond error\n"); + return -ENODEV; + } + + if (status & AM65_CPSW_PN_MAC_VERIFY_ERR) { + netdev_dbg(port->ndev, "MAC Merge verify error\n"); + return -ENODEV; + } + } while (try-- > 0); + + netdev_dbg(port->ndev, "MAC Merge verify timeout\n"); + return -ETIMEDOUT; +} + +static void am65_cpsw_iet_set_preempt_mask(struct am65_cpsw_port *port, u8 preemptible_tcs) +{ + u32 val; + + val = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + val &= ~AM65_CPSW_PN_IET_MAC_PREMPT_MASK; + val |= AM65_CPSW_PN_IET_MAC_SET_PREEMPT(preemptible_tcs); + writel(val, port->port_base + AM65_CPSW_PN_REG_IET_CTRL); +} + +/* enable common IET_ENABLE only if at least 1 port has rx IET enabled. + * UAPI doesn't allow tx enable without rx enable. + */ +void am65_cpsw_iet_common_enable(struct am65_cpsw_common *common) +{ + struct am65_cpsw_port *port; + bool rx_enable = false; + u32 val; + int i; + + for (i = 0; i < common->port_num; i++) { + port = &common->ports[i]; + val = readl(port->port_base + AM65_CPSW_PN_REG_CTL); + rx_enable = !!(val & AM65_CPSW_PN_CTL_IET_PORT_EN); + if (rx_enable) + break; + } + + val = readl(common->cpsw_base + AM65_CPSW_REG_CTL); + + if (rx_enable) + val |= AM65_CPSW_CTL_IET_EN; + else + val &= ~AM65_CPSW_CTL_IET_EN; + + writel(val, common->cpsw_base + AM65_CPSW_REG_CTL); + common->iet_enabled = rx_enable; +} + +/* CPSW does not have an IRQ to notify changes to the MAC Merge TX status + * (active/inactive), but the preemptible traffic classes should only be + * committed to hardware once TX is active. Resort to polling. + */ +void am65_cpsw_iet_commit_preemptible_tcs(struct am65_cpsw_port *port) +{ + u8 preemptible_tcs; + int err; + u32 val; + + if (port->qos.link_speed == SPEED_UNKNOWN) + return; + + val = readl(port->port_base + AM65_CPSW_PN_REG_CTL); + if (!(val & AM65_CPSW_PN_CTL_IET_PORT_EN)) + return; + + /* update common IET enable */ + am65_cpsw_iet_common_enable(port->common); + + /* update verify count */ + err = am65_cpsw_iet_set_verify_timeout_count(port); + if (err) { + netdev_err(port->ndev, "couldn't set verify count: %d\n", err); + return; + } + + val = readl(port->port_base + AM65_CPSW_PN_REG_IET_CTRL); + if (!(val & AM65_CPSW_PN_IET_MAC_DISABLEVERIFY)) { + err = am65_cpsw_iet_verify_wait(port); + if (err) + return; + } + + preemptible_tcs = port->qos.iet.preemptible_tcs; + am65_cpsw_iet_set_preempt_mask(port, preemptible_tcs); +} + +static void am65_cpsw_iet_change_preemptible_tcs(struct am65_cpsw_port *port, u8 preemptible_tcs) +{ + struct am65_cpsw_ndev_priv *priv = am65_ndev_to_priv(port->ndev); + + port->qos.iet.preemptible_tcs = preemptible_tcs; + mutex_lock(&priv->mm_lock); + am65_cpsw_iet_commit_preemptible_tcs(port); + mutex_unlock(&priv->mm_lock); +} + +static void am65_cpsw_iet_link_state_update(struct net_device *ndev) +{ + struct am65_cpsw_ndev_priv *priv = am65_ndev_to_priv(ndev); + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + + mutex_lock(&priv->mm_lock); + am65_cpsw_iet_commit_preemptible_tcs(port); + mutex_unlock(&priv->mm_lock); +} + static int am65_cpsw_port_est_enabled(struct am65_cpsw_port *port) { return port->qos.est_oper || port->qos.est_admin; @@ -428,7 +804,7 @@ static void am65_cpsw_stop_est(struct net_device *ndev) am65_cpsw_timer_stop(ndev); } -static void am65_cpsw_purge_est(struct net_device *ndev) +static void am65_cpsw_taprio_destroy(struct net_device *ndev) { struct am65_cpsw_port *port = am65_ndev_to_port(ndev); @@ -439,31 +815,74 @@ static void am65_cpsw_purge_est(struct net_device *ndev) port->qos.est_oper = NULL; port->qos.est_admin = NULL; + + am65_cpsw_reset_tc_mqprio(ndev); } -static int am65_cpsw_configure_taprio(struct net_device *ndev, - struct am65_cpsw_est *est_new) +static void am65_cpsw_cp_taprio(struct tc_taprio_qopt_offload *from, + struct tc_taprio_qopt_offload *to) +{ + int i; + + *to = *from; + for (i = 0; i < from->num_entries; i++) + to->entries[i] = from->entries[i]; +} + +static int am65_cpsw_taprio_replace(struct net_device *ndev, + struct tc_taprio_qopt_offload *taprio) { struct am65_cpsw_common *common = am65_ndev_to_common(ndev); + struct netlink_ext_ack *extack = taprio->mqprio.extack; + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); struct am65_cpts *cpts = common->cpts; - int ret = 0, tact = TACT_PROG; + struct am65_cpsw_est *est_new; + int ret, tact; - am65_cpsw_est_update_state(ndev); + if (!netif_running(ndev)) { + NL_SET_ERR_MSG_MOD(extack, "interface is down, link speed unknown"); + return -ENETDOWN; + } - if (est_new->taprio.cmd == TAPRIO_CMD_DESTROY) { - am65_cpsw_stop_est(ndev); - return ret; + if (common->pf_p0_rx_ptype_rrobin) { + NL_SET_ERR_MSG_MOD(extack, + "p0-rx-ptype-rrobin flag conflicts with taprio qdisc"); + return -EINVAL; } + if (port->qos.link_speed == SPEED_UNKNOWN) + return -ENOLINK; + + if (taprio->cycle_time_extension) { + NL_SET_ERR_MSG_MOD(extack, + "cycle time extension not supported"); + return -EOPNOTSUPP; + } + + est_new = devm_kzalloc(&ndev->dev, + struct_size(est_new, taprio.entries, taprio->num_entries), + GFP_KERNEL); + if (!est_new) + return -ENOMEM; + + ret = am65_cpsw_setup_mqprio(ndev, &taprio->mqprio); + if (ret) + return ret; + + am65_cpsw_cp_taprio(taprio, &est_new->taprio); + + am65_cpsw_est_update_state(ndev); + ret = am65_cpsw_est_check_scheds(ndev, est_new); if (ret < 0) - return ret; + goto fail; tact = am65_cpsw_timer_act(ndev, est_new); if (tact == TACT_NEED_STOP) { - dev_err(&ndev->dev, - "Can't toggle estf timer, stop taprio first"); - return -EINVAL; + NL_SET_ERR_MSG_MOD(extack, + "Can't toggle estf timer, stop taprio first"); + ret = -EINVAL; + goto fail; } if (tact == TACT_PROG) @@ -476,62 +895,26 @@ static int am65_cpsw_configure_taprio(struct net_device *ndev, am65_cpsw_est_set_sched_list(ndev, est_new); am65_cpsw_port_est_assign_buf_num(ndev, est_new->buf); - am65_cpsw_est_set(ndev, est_new->taprio.cmd == TAPRIO_CMD_REPLACE); + am65_cpsw_est_set(ndev, 1); if (tact == TACT_PROG) { ret = am65_cpsw_timer_set(ndev, est_new); if (ret) { - dev_err(&ndev->dev, "Failed to set cycle time"); - return ret; + NL_SET_ERR_MSG_MOD(extack, + "Failed to set cycle time"); + goto fail; } } - return ret; -} - -static void am65_cpsw_cp_taprio(struct tc_taprio_qopt_offload *from, - struct tc_taprio_qopt_offload *to) -{ - int i; - - *to = *from; - for (i = 0; i < from->num_entries; i++) - to->entries[i] = from->entries[i]; -} - -static int am65_cpsw_set_taprio(struct net_device *ndev, void *type_data) -{ - struct am65_cpsw_port *port = am65_ndev_to_port(ndev); - struct tc_taprio_qopt_offload *taprio = type_data; - struct am65_cpsw_est *est_new; - int ret = 0; - - if (taprio->cycle_time_extension) { - dev_err(&ndev->dev, "Failed to set cycle time extension"); - return -EOPNOTSUPP; - } - - est_new = devm_kzalloc(&ndev->dev, - struct_size(est_new, taprio.entries, taprio->num_entries), - GFP_KERNEL); - if (!est_new) - return -ENOMEM; - - am65_cpsw_cp_taprio(taprio, &est_new->taprio); - ret = am65_cpsw_configure_taprio(ndev, est_new); - if (!ret) { - if (taprio->cmd == TAPRIO_CMD_REPLACE) { - devm_kfree(&ndev->dev, port->qos.est_admin); + devm_kfree(&ndev->dev, port->qos.est_admin); + port->qos.est_admin = est_new; + am65_cpsw_iet_change_preemptible_tcs(port, taprio->mqprio.preemptible_tcs); - port->qos.est_admin = est_new; - } else { - devm_kfree(&ndev->dev, est_new); - am65_cpsw_purge_est(ndev); - } - } else { - devm_kfree(&ndev->dev, est_new); - } + return 0; +fail: + am65_cpsw_reset_tc_mqprio(ndev); + devm_kfree(&ndev->dev, est_new); return ret; } @@ -541,7 +924,6 @@ static void am65_cpsw_est_link_up(struct net_device *ndev, int link_speed) ktime_t cur_time; s64 delta; - port->qos.link_speed = link_speed; if (!am65_cpsw_port_est_enabled(port)) return; @@ -558,37 +940,26 @@ static void am65_cpsw_est_link_up(struct net_device *ndev, int link_speed) return; purge_est: - am65_cpsw_purge_est(ndev); + am65_cpsw_taprio_destroy(ndev); } static int am65_cpsw_setup_taprio(struct net_device *ndev, void *type_data) { - struct am65_cpsw_port *port = am65_ndev_to_port(ndev); struct tc_taprio_qopt_offload *taprio = type_data; - struct am65_cpsw_common *common = port->common; - - if (taprio->cmd != TAPRIO_CMD_REPLACE && - taprio->cmd != TAPRIO_CMD_DESTROY) - return -EOPNOTSUPP; - - if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_TAS)) - return -ENODEV; - - if (!netif_running(ndev)) { - dev_err(&ndev->dev, "interface is down, link speed unknown\n"); - return -ENETDOWN; - } - - if (common->pf_p0_rx_ptype_rrobin) { - dev_err(&ndev->dev, - "p0-rx-ptype-rrobin flag conflicts with taprio qdisc\n"); - return -EINVAL; + int err = 0; + + switch (taprio->cmd) { + case TAPRIO_CMD_REPLACE: + err = am65_cpsw_taprio_replace(ndev, taprio); + break; + case TAPRIO_CMD_DESTROY: + am65_cpsw_taprio_destroy(ndev); + break; + default: + err = -EOPNOTSUPP; } - if (port->qos.link_speed == SPEED_UNKNOWN) - return -ENOLINK; - - return am65_cpsw_set_taprio(ndev, type_data); + return err; } static int am65_cpsw_tc_query_caps(struct net_device *ndev, void *type_data) @@ -596,12 +967,17 @@ static int am65_cpsw_tc_query_caps(struct net_device *ndev, void *type_data) struct tc_query_caps_base *base = type_data; switch (base->type) { + case TC_SETUP_QDISC_MQPRIO: { + struct tc_mqprio_caps *caps = base->caps; + + caps->validate_queue_counts = true; + + return 0; + } + case TC_SETUP_QDISC_TAPRIO: { struct tc_taprio_caps *caps = base->caps; - if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_TAS)) - return -EOPNOTSUPP; - caps->gate_mask_per_txq = true; return 0; @@ -787,55 +1163,6 @@ static int am65_cpsw_qos_setup_tc_block(struct net_device *ndev, struct flow_blo port, port, true); } -int am65_cpsw_qos_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type, - void *type_data) -{ - switch (type) { - case TC_QUERY_CAPS: - return am65_cpsw_tc_query_caps(ndev, type_data); - case TC_SETUP_QDISC_TAPRIO: - return am65_cpsw_setup_taprio(ndev, type_data); - case TC_SETUP_BLOCK: - return am65_cpsw_qos_setup_tc_block(ndev, type_data); - default: - return -EOPNOTSUPP; - } -} - -void am65_cpsw_qos_link_up(struct net_device *ndev, int link_speed) -{ - struct am65_cpsw_port *port = am65_ndev_to_port(ndev); - - if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_TAS)) - return; - - am65_cpsw_est_link_up(ndev, link_speed); - port->qos.link_down_time = 0; -} - -void am65_cpsw_qos_link_down(struct net_device *ndev) -{ - struct am65_cpsw_port *port = am65_ndev_to_port(ndev); - - if (!IS_ENABLED(CONFIG_TI_AM65_CPSW_TAS)) - return; - - if (!port->qos.link_down_time) - port->qos.link_down_time = ktime_get(); - - port->qos.link_speed = SPEED_UNKNOWN; -} - -static u32 -am65_cpsw_qos_tx_rate_calc(u32 rate_mbps, unsigned long bus_freq) -{ - u32 ir; - - bus_freq /= 1000000; - ir = DIV_ROUND_UP(((u64)rate_mbps * 32768), bus_freq); - return ir; -} - static void am65_cpsw_qos_tx_p0_rate_apply(struct am65_cpsw_common *common, int tx_ch, u32 rate_mbps) @@ -937,3 +1264,44 @@ void am65_cpsw_qos_tx_p0_rate_init(struct am65_cpsw_common *common) host->port_base + AM65_CPSW_PN_REG_PRI_CIR(tx_ch)); } } + +int am65_cpsw_qos_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type, + void *type_data) +{ + switch (type) { + case TC_QUERY_CAPS: + return am65_cpsw_tc_query_caps(ndev, type_data); + case TC_SETUP_QDISC_TAPRIO: + return am65_cpsw_setup_taprio(ndev, type_data); + case TC_SETUP_QDISC_MQPRIO: + return am65_cpsw_setup_mqprio(ndev, type_data); + case TC_SETUP_BLOCK: + return am65_cpsw_qos_setup_tc_block(ndev, type_data); + default: + return -EOPNOTSUPP; + } +} + +void am65_cpsw_qos_link_up(struct net_device *ndev, int link_speed) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + + port->qos.link_speed = link_speed; + am65_cpsw_tx_pn_shaper_apply(port); + am65_cpsw_iet_link_state_update(ndev); + + am65_cpsw_est_link_up(ndev, link_speed); + port->qos.link_down_time = 0; +} + +void am65_cpsw_qos_link_down(struct net_device *ndev) +{ + struct am65_cpsw_port *port = am65_ndev_to_port(ndev); + + port->qos.link_speed = SPEED_UNKNOWN; + am65_cpsw_tx_pn_shaper_apply(port); + am65_cpsw_iet_link_state_update(ndev); + + if (!port->qos.link_down_time) + port->qos.link_down_time = ktime_get(); +} diff --git a/drivers/net/ethernet/ti/am65-cpsw-qos.h b/drivers/net/ethernet/ti/am65-cpsw-qos.h index 0cc2a3b3d7f9..b328e56c5b2b 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-qos.h +++ b/drivers/net/ethernet/ti/am65-cpsw-qos.h @@ -9,6 +9,7 @@ #include <net/pkt_sched.h> struct am65_cpsw_common; +struct am65_cpsw_port; struct am65_cpsw_est { int buf; @@ -16,6 +17,18 @@ struct am65_cpsw_est { struct tc_taprio_qopt_offload taprio; }; +struct am65_cpsw_mqprio { + struct tc_mqprio_qopt_offload mqprio_hw; + u64 max_rate_total; + bool shaper_en; +}; + +struct am65_cpsw_iet { + u8 preemptible_tcs; + u32 original_max_blks; + int verify_time_ms; +}; + struct am65_cpsw_ale_ratelimit { unsigned long cookie; u64 rate_packet_ps; @@ -26,16 +39,189 @@ struct am65_cpsw_qos { struct am65_cpsw_est *est_oper; ktime_t link_down_time; int link_speed; + struct am65_cpsw_mqprio mqprio; + struct am65_cpsw_iet iet; struct am65_cpsw_ale_ratelimit ale_bc_ratelimit; struct am65_cpsw_ale_ratelimit ale_mc_ratelimit; }; +#define AM65_CPSW_REG_CTL 0x004 +#define AM65_CPSW_PN_REG_CTL 0x004 +#define AM65_CPSW_PN_REG_FIFO_STATUS 0x050 +#define AM65_CPSW_PN_REG_EST_CTL 0x060 +#define AM65_CPSW_PN_REG_PRI_CIR(pri) (0x140 + 4 * (pri)) +#define AM65_CPSW_P0_REG_PRI_EIR(pri) (0x160 + 4 * (pri)) + +#define AM65_CPSW_PN_REG_CTL 0x004 +#define AM65_CPSW_PN_REG_TX_PRI_MAP 0x018 +#define AM65_CPSW_PN_REG_RX_PRI_MAP 0x020 +#define AM65_CPSW_PN_REG_FIFO_STATUS 0x050 +#define AM65_CPSW_PN_REG_EST_CTL 0x060 +#define AM65_CPSW_PN_REG_PRI_CIR(pri) (0x140 + 4 * (pri)) +#define AM65_CPSW_PN_REG_PRI_EIR(pri) (0x160 + 4 * (pri)) + +/* AM65_CPSW_REG_CTL register fields */ +#define AM65_CPSW_CTL_EST_EN BIT(18) + +/* AM65_CPSW_PN_REG_CTL register fields */ +#define AM65_CPSW_PN_CTL_EST_PORT_EN BIT(17) + +/* AM65_CPSW_PN_REG_EST_CTL register fields */ +#define AM65_CPSW_PN_EST_ONEBUF BIT(0) +#define AM65_CPSW_PN_EST_BUFSEL BIT(1) +#define AM65_CPSW_PN_EST_TS_EN BIT(2) +#define AM65_CPSW_PN_EST_TS_FIRST BIT(3) +#define AM65_CPSW_PN_EST_ONEPRI BIT(4) +#define AM65_CPSW_PN_EST_TS_PRI_MSK GENMASK(7, 5) + +/* AM65_CPSW_PN_REG_FIFO_STATUS register fields */ +#define AM65_CPSW_PN_FST_TX_PRI_ACTIVE_MSK GENMASK(7, 0) +#define AM65_CPSW_PN_FST_TX_E_MAC_ALLOW_MSK GENMASK(15, 8) +#define AM65_CPSW_PN_FST_EST_CNT_ERR BIT(16) +#define AM65_CPSW_PN_FST_EST_ADD_ERR BIT(17) +#define AM65_CPSW_PN_FST_EST_BUFACT BIT(18) + +/* EST FETCH COMMAND RAM */ +#define AM65_CPSW_FETCH_RAM_CMD_NUM 0x80 +#define AM65_CPSW_FETCH_CNT_MSK GENMASK(21, 8) +#define AM65_CPSW_FETCH_CNT_MAX (AM65_CPSW_FETCH_CNT_MSK >> 8) +#define AM65_CPSW_FETCH_CNT_OFFSET 8 +#define AM65_CPSW_FETCH_ALLOW_MSK GENMASK(7, 0) +#define AM65_CPSW_FETCH_ALLOW_MAX AM65_CPSW_FETCH_ALLOW_MSK + +/* number of priority queues per port FIFO */ +#define AM65_CPSW_PN_FIFO_PRIO_NUM 8 + +#if IS_ENABLED(CONFIG_TI_AM65_CPSW_QOS) int am65_cpsw_qos_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type, void *type_data); void am65_cpsw_qos_link_up(struct net_device *ndev, int link_speed); void am65_cpsw_qos_link_down(struct net_device *ndev); int am65_cpsw_qos_ndo_tx_p0_set_maxrate(struct net_device *ndev, int queue, u32 rate_mbps); void am65_cpsw_qos_tx_p0_rate_init(struct am65_cpsw_common *common); +void am65_cpsw_iet_commit_preemptible_tcs(struct am65_cpsw_port *port); +void am65_cpsw_iet_common_enable(struct am65_cpsw_common *common); +#else +static inline int am65_cpsw_qos_ndo_setup_tc(struct net_device *ndev, + enum tc_setup_type type, + void *type_data) +{ + return -EOPNOTSUPP; +} + +static inline void am65_cpsw_qos_link_up(struct net_device *ndev, + int link_speed) +{ } + +static inline void am65_cpsw_qos_link_down(struct net_device *ndev) +{ } + +static inline int am65_cpsw_qos_ndo_tx_p0_set_maxrate(struct net_device *ndev, + int queue, + u32 rate_mbps) +{ + return 0; +} + +static inline void am65_cpsw_qos_tx_p0_rate_init(struct am65_cpsw_common *common) +{ } +static inline void am65_cpsw_iet_commit_preemptible_tcs(struct am65_cpsw_port *port) +{ } +static inline void am65_cpsw_iet_common_enable(struct am65_cpsw_common *common) +{ } +#endif + +#define AM65_CPSW_REG_CTL 0x004 +#define AM65_CPSW_PN_REG_CTL 0x004 +#define AM65_CPSW_PN_REG_MAX_BLKS 0x008 +#define AM65_CPSW_PN_REG_TX_PRI_MAP 0x018 +#define AM65_CPSW_PN_REG_RX_PRI_MAP 0x020 +#define AM65_CPSW_PN_REG_IET_CTRL 0x040 +#define AM65_CPSW_PN_REG_IET_STATUS 0x044 +#define AM65_CPSW_PN_REG_IET_VERIFY 0x048 +#define AM65_CPSW_PN_REG_FIFO_STATUS 0x050 +#define AM65_CPSW_PN_REG_EST_CTL 0x060 +#define AM65_CPSW_PN_REG_PRI_CIR(pri) (0x140 + 4 * (pri)) +#define AM65_CPSW_PN_REG_PRI_EIR(pri) (0x160 + 4 * (pri)) + +/* AM65_CPSW_REG_CTL register fields */ +#define AM65_CPSW_CTL_IET_EN BIT(17) +#define AM65_CPSW_CTL_EST_EN BIT(18) + +/* AM65_CPSW_PN_REG_CTL register fields */ +#define AM65_CPSW_PN_CTL_IET_PORT_EN BIT(16) +#define AM65_CPSW_PN_CTL_EST_PORT_EN BIT(17) + +/* AM65_CPSW_PN_REG_EST_CTL register fields */ +#define AM65_CPSW_PN_EST_ONEBUF BIT(0) +#define AM65_CPSW_PN_EST_BUFSEL BIT(1) +#define AM65_CPSW_PN_EST_TS_EN BIT(2) +#define AM65_CPSW_PN_EST_TS_FIRST BIT(3) +#define AM65_CPSW_PN_EST_ONEPRI BIT(4) +#define AM65_CPSW_PN_EST_TS_PRI_MSK GENMASK(7, 5) + +/* AM65_CPSW_PN_REG_IET_CTRL register fields */ +#define AM65_CPSW_PN_IET_MAC_PENABLE BIT(0) +#define AM65_CPSW_PN_IET_MAC_DISABLEVERIFY BIT(2) +#define AM65_CPSW_PN_IET_MAC_LINKFAIL BIT(3) +#define AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_MASK GENMASK(10, 8) +#define AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_OFFSET 8 +#define AM65_CPSW_PN_IET_MAC_PREMPT_MASK GENMASK(23, 16) +#define AM65_CPSW_PN_IET_MAC_PREMPT_OFFSET 16 + +#define AM65_CPSW_PN_IET_MAC_SET_ADDFRAGSIZE(n) (((n) << AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_OFFSET) & \ + AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_MASK) +#define AM65_CPSW_PN_IET_MAC_GET_ADDFRAGSIZE(n) (((n) & AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_MASK) >> \ + AM65_CPSW_PN_IET_MAC_MAC_ADDFRAGSIZE_OFFSET) +#define AM65_CPSW_PN_IET_MAC_SET_PREEMPT(n) (((n) << AM65_CPSW_PN_IET_MAC_PREMPT_OFFSET) & \ + AM65_CPSW_PN_IET_MAC_PREMPT_MASK) +#define AM65_CPSW_PN_IET_MAC_GET_PREEMPT(n) (((n) & AM65_CPSW_PN_IET_MAC_PREMPT_MASK) >> \ + AM65_CPSW_PN_IET_MAC_PREMPT_OFFSET) + +/* AM65_CPSW_PN_REG_IET_STATUS register fields */ +#define AM65_CPSW_PN_MAC_STATUS GENMASK(3, 0) +#define AM65_CPSW_PN_MAC_VERIFIED BIT(0) +#define AM65_CPSW_PN_MAC_VERIFY_FAIL BIT(1) +#define AM65_CPSW_PN_MAC_RESPOND_ERR BIT(2) +#define AM65_CPSW_PN_MAC_VERIFY_ERR BIT(3) + +/* AM65_CPSW_PN_REG_IET_VERIFY register fields */ +#define AM65_CPSW_PN_MAC_VERIFY_CNT_MASK GENMASK(23, 0) +#define AM65_CPSW_PN_MAC_GET_VERIFY_CNT(n) ((n) & AM65_CPSW_PN_MAC_VERIFY_CNT_MASK) +/* 10 msec converted to NSEC */ +#define AM65_CPSW_IET_VERIFY_CNT_MS (10) +#define AM65_CPSW_IET_VERIFY_CNT_NS (AM65_CPSW_IET_VERIFY_CNT_MS * \ + NSEC_PER_MSEC) + +/* AM65_CPSW_PN_REG_FIFO_STATUS register fields */ +#define AM65_CPSW_PN_FST_TX_PRI_ACTIVE_MSK GENMASK(7, 0) +#define AM65_CPSW_PN_FST_TX_E_MAC_ALLOW_MSK GENMASK(15, 8) +#define AM65_CPSW_PN_FST_EST_CNT_ERR BIT(16) +#define AM65_CPSW_PN_FST_EST_ADD_ERR BIT(17) +#define AM65_CPSW_PN_FST_EST_BUFACT BIT(18) + +/* EST FETCH COMMAND RAM */ +#define AM65_CPSW_FETCH_RAM_CMD_NUM 0x80 +#define AM65_CPSW_FETCH_CNT_MSK GENMASK(21, 8) +#define AM65_CPSW_FETCH_CNT_MAX (AM65_CPSW_FETCH_CNT_MSK >> 8) +#define AM65_CPSW_FETCH_CNT_OFFSET 8 +#define AM65_CPSW_FETCH_ALLOW_MSK GENMASK(7, 0) +#define AM65_CPSW_FETCH_ALLOW_MAX AM65_CPSW_FETCH_ALLOW_MSK + +/* AM65_CPSW_PN_REG_MAX_BLKS fields for IET and No IET cases */ +/* 7 blocks for pn_rx_max_blks, 13 for pn_tx_max_blks*/ +#define AM65_CPSW_PN_TX_RX_MAX_BLKS_IET 0xD07 + +/* Slave IET Stats. register offsets */ +#define AM65_CPSW_STATN_IET_RX_ASSEMBLY_ERROR 0x140 +#define AM65_CPSW_STATN_IET_RX_ASSEMBLY_OK 0x144 +#define AM65_CPSW_STATN_IET_RX_SMD_ERROR 0x148 +#define AM65_CPSW_STATN_IET_RX_FRAG 0x14c +#define AM65_CPSW_STATN_IET_TX_HOLD 0x150 +#define AM65_CPSW_STATN_IET_TX_FRAG 0x154 + +/* number of priority queues per port FIFO */ +#define AM65_CPSW_PN_FIFO_PRIO_NUM 8 #endif /* AM65_CPSW_QOS_H_ */ diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 9663050a852d..e34816638569 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -93,6 +93,8 @@ struct pcpu_secy_stats { * @secys: linked list of SecY's on the underlying device * @gro_cells: pointer to the Generic Receive Offload cell * @offload: status of offloading on the MACsec device + * @insert_tx_tag: when offloading, device requires to insert an + * additional tag */ struct macsec_dev { struct macsec_secy secy; @@ -102,6 +104,7 @@ struct macsec_dev { struct list_head secys; struct gro_cells gro_cells; enum macsec_offload offload; + bool insert_tx_tag; }; /** @@ -604,26 +607,11 @@ static struct sk_buff *macsec_encrypt(struct sk_buff *skb, return ERR_PTR(-EINVAL); } - if (unlikely(skb_headroom(skb) < MACSEC_NEEDED_HEADROOM || - skb_tailroom(skb) < MACSEC_NEEDED_TAILROOM)) { - struct sk_buff *nskb = skb_copy_expand(skb, - MACSEC_NEEDED_HEADROOM, - MACSEC_NEEDED_TAILROOM, - GFP_ATOMIC); - if (likely(nskb)) { - consume_skb(skb); - skb = nskb; - } else { - macsec_txsa_put(tx_sa); - kfree_skb(skb); - return ERR_PTR(-ENOMEM); - } - } else { - skb = skb_unshare(skb, GFP_ATOMIC); - if (!skb) { - macsec_txsa_put(tx_sa); - return ERR_PTR(-ENOMEM); - } + ret = skb_ensure_writable_head_tail(skb, dev); + if (unlikely(ret < 0)) { + macsec_txsa_put(tx_sa); + kfree_skb(skb); + return ERR_PTR(ret); } unprotected_len = skb->len; @@ -2583,6 +2571,33 @@ static bool macsec_is_configured(struct macsec_dev *macsec) return false; } +static bool macsec_needs_tx_tag(struct macsec_dev *macsec, + const struct macsec_ops *ops) +{ + return macsec->offload == MACSEC_OFFLOAD_PHY && + ops->mdo_insert_tx_tag; +} + +static void macsec_set_head_tail_room(struct net_device *dev) +{ + struct macsec_dev *macsec = macsec_priv(dev); + struct net_device *real_dev = macsec->real_dev; + int needed_headroom, needed_tailroom; + const struct macsec_ops *ops; + + ops = macsec_get_ops(macsec, NULL); + if (ops) { + needed_headroom = ops->needed_headroom; + needed_tailroom = ops->needed_tailroom; + } else { + needed_headroom = MACSEC_NEEDED_HEADROOM; + needed_tailroom = MACSEC_NEEDED_TAILROOM; + } + + dev->needed_headroom = real_dev->needed_headroom + needed_headroom; + dev->needed_tailroom = real_dev->needed_tailroom + needed_tailroom; +} + static int macsec_update_offload(struct net_device *dev, enum macsec_offload offload) { enum macsec_offload prev_offload; @@ -2620,8 +2635,13 @@ static int macsec_update_offload(struct net_device *dev, enum macsec_offload off ctx.secy = &macsec->secy; ret = offload == MACSEC_OFFLOAD_OFF ? macsec_offload(ops->mdo_del_secy, &ctx) : macsec_offload(ops->mdo_add_secy, &ctx); - if (ret) + if (ret) { macsec->offload = prev_offload; + return ret; + } + + macsec_set_head_tail_room(dev); + macsec->insert_tx_tag = macsec_needs_tx_tag(macsec, ops); return ret; } @@ -3379,6 +3399,40 @@ static struct genl_family macsec_fam __ro_after_init = { .resv_start_op = MACSEC_CMD_UPD_OFFLOAD + 1, }; +static struct sk_buff *macsec_insert_tx_tag(struct sk_buff *skb, + struct net_device *dev) +{ + struct macsec_dev *macsec = macsec_priv(dev); + const struct macsec_ops *ops; + struct phy_device *phydev; + struct macsec_context ctx; + int skb_final_len; + int err; + + ops = macsec_get_ops(macsec, &ctx); + skb_final_len = skb->len - ETH_HLEN + ops->needed_headroom + + ops->needed_tailroom; + if (unlikely(skb_final_len > macsec->real_dev->mtu)) { + err = -EINVAL; + goto cleanup; + } + + phydev = macsec->real_dev->phydev; + + err = skb_ensure_writable_head_tail(skb, dev); + if (unlikely(err < 0)) + goto cleanup; + + err = ops->mdo_insert_tx_tag(phydev, skb); + if (unlikely(err)) + goto cleanup; + + return skb; +cleanup: + kfree_skb(skb); + return ERR_PTR(err); +} + static netdev_tx_t macsec_start_xmit(struct sk_buff *skb, struct net_device *dev) { @@ -3393,6 +3447,15 @@ static netdev_tx_t macsec_start_xmit(struct sk_buff *skb, skb_dst_drop(skb); dst_hold(&md_dst->dst); skb_dst_set(skb, &md_dst->dst); + + if (macsec->insert_tx_tag) { + skb = macsec_insert_tx_tag(skb, dev); + if (IS_ERR(skb)) { + DEV_STATS_INC(dev, tx_dropped); + return NETDEV_TX_OK; + } + } + skb->dev = macsec->real_dev; return dev_queue_xmit(skb); } @@ -3454,10 +3517,7 @@ static int macsec_dev_init(struct net_device *dev) dev->features = real_dev->features & MACSEC_FEATURES; dev->features |= NETIF_F_LLTX | NETIF_F_GSO_SOFTWARE; - dev->needed_headroom = real_dev->needed_headroom + - MACSEC_NEEDED_HEADROOM; - dev->needed_tailroom = real_dev->needed_tailroom + - MACSEC_NEEDED_TAILROOM; + macsec_set_head_tail_room(dev); if (is_zero_ether_addr(dev->dev_addr)) eth_hw_addr_inherit(dev, real_dev); @@ -3604,21 +3664,19 @@ static int macsec_set_mac_address(struct net_device *dev, void *p) struct macsec_dev *macsec = macsec_priv(dev); struct net_device *real_dev = macsec->real_dev; struct sockaddr *addr = p; + u8 old_addr[ETH_ALEN]; int err; if (!is_valid_ether_addr(addr->sa_data)) return -EADDRNOTAVAIL; - if (!(dev->flags & IFF_UP)) - goto out; - - err = dev_uc_add(real_dev, addr->sa_data); - if (err < 0) - return err; - - dev_uc_del(real_dev, dev->dev_addr); + if (dev->flags & IFF_UP) { + err = dev_uc_add(real_dev, addr->sa_data); + if (err < 0) + return err; + } -out: + ether_addr_copy(old_addr, dev->dev_addr); eth_hw_addr_set(dev, addr->sa_data); /* If h/w offloading is available, propagate to the device */ @@ -3627,13 +3685,29 @@ out: struct macsec_context ctx; ops = macsec_get_ops(macsec, &ctx); - if (ops) { - ctx.secy = &macsec->secy; - macsec_offload(ops->mdo_upd_secy, &ctx); + if (!ops) { + err = -EOPNOTSUPP; + goto restore_old_addr; } + + ctx.secy = &macsec->secy; + err = macsec_offload(ops->mdo_upd_secy, &ctx); + if (err) + goto restore_old_addr; } + if (dev->flags & IFF_UP) + dev_uc_del(real_dev, old_addr); + return 0; + +restore_old_addr: + if (dev->flags & IFF_UP) + dev_uc_del(real_dev, addr->sa_data); + + eth_hw_addr_set(dev, old_addr); + + return err; } static int macsec_change_mtu(struct net_device *dev, int new_mtu) @@ -4126,6 +4200,9 @@ static int macsec_newlink(struct net *net, struct net_device *dev, err = macsec_offload(ops->mdo_add_secy, &ctx); if (err) goto del_dev; + + macsec->insert_tx_tag = + macsec_needs_tx_tag(macsec, ops); } } diff --git a/drivers/net/mdio/of_mdio.c b/drivers/net/mdio/of_mdio.c index 64ebcb6d235c..9b6cab6154e0 100644 --- a/drivers/net/mdio/of_mdio.c +++ b/drivers/net/mdio/of_mdio.c @@ -139,6 +139,11 @@ bool of_mdiobus_child_is_phy(struct device_node *child) } EXPORT_SYMBOL(of_mdiobus_child_is_phy); +static void __of_mdiobus_unregister_callback(struct mii_bus *mdio) +{ + of_node_put(mdio->dev.of_node); +} + /** * __of_mdiobus_register - Register mii_bus and create PHYs from the device tree * @mdio: pointer to mii_bus structure @@ -166,6 +171,8 @@ int __of_mdiobus_register(struct mii_bus *mdio, struct device_node *np, * the device tree are populated after the bus has been registered */ mdio->phy_mask = ~0; + mdio->__unregister_callback = __of_mdiobus_unregister_callback; + of_node_get(np); device_set_node(&mdio->dev, of_fwnode_handle(np)); /* Get bus level PHY reset GPIO details */ @@ -177,7 +184,7 @@ int __of_mdiobus_register(struct mii_bus *mdio, struct device_node *np, /* Register the MDIO bus */ rc = __mdiobus_register(mdio, owner); if (rc) - return rc; + goto put_node; /* Loop over the child nodes and register a phy_device for each phy */ for_each_available_child_of_node(np, child) { @@ -237,6 +244,9 @@ int __of_mdiobus_register(struct mii_bus *mdio, struct device_node *np, unregister: of_node_put(child); mdiobus_unregister(mdio); + +put_node: + of_node_put(np); return rc; } EXPORT_SYMBOL(__of_mdiobus_register); diff --git a/drivers/net/netdevsim/macsec.c b/drivers/net/netdevsim/macsec.c index 0d5f50430dd3..aa007b1e4b78 100644 --- a/drivers/net/netdevsim/macsec.c +++ b/drivers/net/netdevsim/macsec.c @@ -3,11 +3,6 @@ #include <net/macsec.h> #include "netdevsim.h" -static inline u64 sci_to_cpu(sci_t sci) -{ - return be64_to_cpu((__force __be64)sci); -} - static int nsim_macsec_find_secy(struct netdevsim *ns, sci_t sci) { int i; diff --git a/drivers/net/phy/Kconfig b/drivers/net/phy/Kconfig index 2e4667bf9ff5..9e2672800f0b 100644 --- a/drivers/net/phy/Kconfig +++ b/drivers/net/phy/Kconfig @@ -317,9 +317,10 @@ config NXP_CBTX_PHY config NXP_C45_TJA11XX_PHY tristate "NXP C45 TJA11XX PHYs" depends on PTP_1588_CLOCK_OPTIONAL + depends on MACSEC || !MACSEC help Enable support for NXP C45 TJA11XX PHYs. - Currently supports the TJA1103 and TJA1120 PHYs. + Currently supports the TJA1103, TJA1104 and TJA1120 PHYs. config NXP_TJA11XX_PHY tristate "NXP TJA11xx PHYs support" diff --git a/drivers/net/phy/Makefile b/drivers/net/phy/Makefile index e35ea69d9cb4..f218954fd7a8 100644 --- a/drivers/net/phy/Makefile +++ b/drivers/net/phy/Makefile @@ -2,7 +2,7 @@ # Makefile for Linux PHY drivers libphy-y := phy.o phy-c45.o phy-core.o phy_device.o \ - linkmode.o + linkmode.o phy_link_topology.o mdio-bus-y += mdio_bus.o mdio_device.o ifdef CONFIG_MDIO_DEVICE @@ -84,7 +84,11 @@ obj-$(CONFIG_MICROSEMI_PHY) += mscc/ obj-$(CONFIG_MOTORCOMM_PHY) += motorcomm.o obj-$(CONFIG_NATIONAL_PHY) += national.o obj-$(CONFIG_NCN26000_PHY) += ncn26000.o -obj-$(CONFIG_NXP_C45_TJA11XX_PHY) += nxp-c45-tja11xx.o +nxp-c45-tja-objs += nxp-c45-tja11xx.o +ifdef CONFIG_MACSEC +nxp-c45-tja-objs += nxp-c45-tja11xx-macsec.o +endif +obj-$(CONFIG_NXP_C45_TJA11XX_PHY) += nxp-c45-tja.o obj-$(CONFIG_NXP_CBTX_PHY) += nxp-cbtx.o obj-$(CONFIG_NXP_TJA11XX_PHY) += nxp-tja11xx.o obj-$(CONFIG_QSEMI_PHY) += qsemi.o diff --git a/drivers/net/phy/at803x.c b/drivers/net/phy/at803x.c index d5dc927618ab..aaf6c654aaed 100644 --- a/drivers/net/phy/at803x.c +++ b/drivers/net/phy/at803x.c @@ -1452,6 +1452,8 @@ static const struct sfp_upstream_ops at8031_sfp_ops = { .attach = phy_sfp_attach, .detach = phy_sfp_detach, .module_insert = at8031_sfp_insert, + .connect_phy = phy_sfp_connect_phy, + .disconnect_phy = phy_sfp_disconnect_phy, }; static int at8031_parse_dt(struct phy_device *phydev) @@ -1781,27 +1783,27 @@ static int qca808x_phy_fast_retrain_config(struct phy_device *phydev) return ret; phy_write_mmd(phydev, MDIO_MMD_AN, QCA808X_PHY_MMD7_TOP_OPTION1, - QCA808X_TOP_OPTION1_DATA); + QCA808X_TOP_OPTION1_DATA); phy_write_mmd(phydev, MDIO_MMD_PMAPMD, QCA808X_PHY_MMD1_MSE_THRESHOLD_20DB, - QCA808X_MSE_THRESHOLD_20DB_VALUE); + QCA808X_MSE_THRESHOLD_20DB_VALUE); phy_write_mmd(phydev, MDIO_MMD_PMAPMD, QCA808X_PHY_MMD1_MSE_THRESHOLD_17DB, - QCA808X_MSE_THRESHOLD_17DB_VALUE); + QCA808X_MSE_THRESHOLD_17DB_VALUE); phy_write_mmd(phydev, MDIO_MMD_PMAPMD, QCA808X_PHY_MMD1_MSE_THRESHOLD_27DB, - QCA808X_MSE_THRESHOLD_27DB_VALUE); + QCA808X_MSE_THRESHOLD_27DB_VALUE); phy_write_mmd(phydev, MDIO_MMD_PMAPMD, QCA808X_PHY_MMD1_MSE_THRESHOLD_28DB, - QCA808X_MSE_THRESHOLD_28DB_VALUE); + QCA808X_MSE_THRESHOLD_28DB_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_1, - QCA808X_MMD3_DEBUG_1_VALUE); + QCA808X_MMD3_DEBUG_1_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_4, - QCA808X_MMD3_DEBUG_4_VALUE); + QCA808X_MMD3_DEBUG_4_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_5, - QCA808X_MMD3_DEBUG_5_VALUE); + QCA808X_MMD3_DEBUG_5_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_3, - QCA808X_MMD3_DEBUG_3_VALUE); + QCA808X_MMD3_DEBUG_3_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_6, - QCA808X_MMD3_DEBUG_6_VALUE); + QCA808X_MMD3_DEBUG_6_VALUE); phy_write_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_DEBUG_2, - QCA808X_MMD3_DEBUG_2_VALUE); + QCA808X_MMD3_DEBUG_2_VALUE); return 0; } @@ -1838,13 +1840,14 @@ static int qca808x_config_init(struct phy_device *phydev) /* Active adc&vga on 802.3az for the link 1000M and 100M */ ret = phy_modify_mmd(phydev, MDIO_MMD_PCS, QCA808X_PHY_MMD3_ADDR_CLD_CTRL7, - QCA808X_8023AZ_AFE_CTRL_MASK, QCA808X_8023AZ_AFE_EN); + QCA808X_8023AZ_AFE_CTRL_MASK, QCA808X_8023AZ_AFE_EN); if (ret) return ret; /* Adjust the threshold on 802.3az for the link 1000M */ ret = phy_write_mmd(phydev, MDIO_MMD_PCS, - QCA808X_PHY_MMD3_AZ_TRAINING_CTRL, QCA808X_MMD3_AZ_TRAINING_VAL); + QCA808X_PHY_MMD3_AZ_TRAINING_CTRL, + QCA808X_MMD3_AZ_TRAINING_VAL); if (ret) return ret; @@ -1870,7 +1873,8 @@ static int qca808x_config_init(struct phy_device *phydev) /* Configure adc threshold as 100mv for the link 10M */ return at803x_debug_reg_mask(phydev, QCA808X_PHY_DEBUG_ADC_THRESHOLD, - QCA808X_ADC_THRESHOLD_MASK, QCA808X_ADC_THRESHOLD_100MV); + QCA808X_ADC_THRESHOLD_MASK, + QCA808X_ADC_THRESHOLD_100MV); } static int qca808x_read_status(struct phy_device *phydev) @@ -1883,7 +1887,7 @@ static int qca808x_read_status(struct phy_device *phydev) return ret; linkmode_mod_bit(ETHTOOL_LINK_MODE_2500baseT_Full_BIT, phydev->lp_advertising, - ret & MDIO_AN_10GBT_STAT_LP2_5G); + ret & MDIO_AN_10GBT_STAT_LP2_5G); ret = genphy_read_status(phydev); if (ret) @@ -1913,7 +1917,7 @@ static int qca808x_read_status(struct phy_device *phydev) */ if (qca808x_has_fast_retrain_or_slave_seed(phydev)) { if (phydev->master_slave_state == MASTER_SLAVE_STATE_ERR || - qca808x_is_prefer_master(phydev)) { + qca808x_is_prefer_master(phydev)) { qca808x_phy_ms_seed_enable(phydev, false); } else { qca808x_phy_ms_seed_enable(phydev, true); @@ -2070,18 +2074,22 @@ static int qca808x_cable_test_get_status(struct phy_device *phydev, bool *finish ethnl_cable_test_result(phydev, ETHTOOL_A_CABLE_PAIR_D, qca808x_cable_test_result_trans(pair_d)); - if (qca808x_cdt_fault_length_valid(pair_a)) - ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_A, - qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_A)); - if (qca808x_cdt_fault_length_valid(pair_b)) - ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_B, - qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_B)); - if (qca808x_cdt_fault_length_valid(pair_c)) - ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_C, - qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_C)); - if (qca808x_cdt_fault_length_valid(pair_d)) - ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_D, - qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_D)); + if (qca808x_cdt_fault_length_valid(pair_a)) { + val = qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_A); + ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_A, val); + } + if (qca808x_cdt_fault_length_valid(pair_b)) { + val = qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_B); + ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_B, val); + } + if (qca808x_cdt_fault_length_valid(pair_c)) { + val = qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_C); + ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_C, val); + } + if (qca808x_cdt_fault_length_valid(pair_d)) { + val = qca808x_cdt_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_D); + ethnl_cable_test_fault_length(phydev, ETHTOOL_A_CABLE_PAIR_D, val); + } *finished = true; @@ -2148,8 +2156,9 @@ static void qca808x_link_change_notify(struct phy_device *phydev) * the interface device address is always phy address added by 1. */ mdiobus_c45_modify_changed(phydev->mdio.bus, phydev->mdio.addr + 1, - MDIO_MMD_PMAPMD, QCA8081_PHY_SERDES_MMD1_FIFO_CTRL, - QCA8081_PHY_FIFO_RSTN, phydev->link ? QCA8081_PHY_FIFO_RSTN : 0); + MDIO_MMD_PMAPMD, QCA8081_PHY_SERDES_MMD1_FIFO_CTRL, + QCA8081_PHY_FIFO_RSTN, + phydev->link ? QCA8081_PHY_FIFO_RSTN : 0); } static struct phy_driver at803x_driver[] = { diff --git a/drivers/net/phy/marvell-88x2222.c b/drivers/net/phy/marvell-88x2222.c index e3aa30dad2e6..3f77bbc7e04f 100644 --- a/drivers/net/phy/marvell-88x2222.c +++ b/drivers/net/phy/marvell-88x2222.c @@ -555,6 +555,8 @@ static const struct sfp_upstream_ops sfp_phy_ops = { .link_down = mv2222_sfp_link_down, .attach = phy_sfp_attach, .detach = phy_sfp_detach, + .connect_phy = phy_sfp_connect_phy, + .disconnect_phy = phy_sfp_disconnect_phy, }; static int mv2222_probe(struct phy_device *phydev) diff --git a/drivers/net/phy/marvell.c b/drivers/net/phy/marvell.c index eba652a4c1d8..674e29bce2cc 100644 --- a/drivers/net/phy/marvell.c +++ b/drivers/net/phy/marvell.c @@ -3254,6 +3254,8 @@ static const struct sfp_upstream_ops m88e1510_sfp_ops = { .module_remove = m88e1510_sfp_remove, .attach = phy_sfp_attach, .detach = phy_sfp_detach, + .connect_phy = phy_sfp_connect_phy, + .disconnect_phy = phy_sfp_disconnect_phy, }; static int m88e1510_probe(struct phy_device *phydev) diff --git a/drivers/net/phy/marvell10g.c b/drivers/net/phy/marvell10g.c index ad43e280930c..6642eb642d4b 100644 --- a/drivers/net/phy/marvell10g.c +++ b/drivers/net/phy/marvell10g.c @@ -503,6 +503,8 @@ static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) static const struct sfp_upstream_ops mv3310_sfp_ops = { .attach = phy_sfp_attach, .detach = phy_sfp_detach, + .connect_phy = phy_sfp_connect_phy, + .disconnect_phy = phy_sfp_disconnect_phy, .module_insert = mv3310_sfp_insert, }; diff --git a/drivers/net/phy/mdio_bus.c b/drivers/net/phy/mdio_bus.c index 6cf73c15635b..4a30757c4ff8 100644 --- a/drivers/net/phy/mdio_bus.c +++ b/drivers/net/phy/mdio_bus.c @@ -787,6 +787,9 @@ void mdiobus_unregister(struct mii_bus *bus) gpiod_set_value_cansleep(bus->reset_gpiod, 1); device_del(&bus->dev); + + if (bus->__unregister_callback) + bus->__unregister_callback(bus); } EXPORT_SYMBOL(mdiobus_unregister); diff --git a/drivers/net/phy/nxp-c45-tja11xx-macsec.c b/drivers/net/phy/nxp-c45-tja11xx-macsec.c new file mode 100644 index 000000000000..550ef08970f4 --- /dev/null +++ b/drivers/net/phy/nxp-c45-tja11xx-macsec.c @@ -0,0 +1,1729 @@ +// SPDX-License-Identifier: GPL-2.0 +/* NXP C45 PTP PHY driver interface + * Copyright 2023 NXP + * Author: Radu Pirea <[email protected]> + */ + +#include <linux/delay.h> +#include <linux/ethtool_netlink.h> +#include <linux/kernel.h> +#include <linux/mii.h> +#include <linux/module.h> +#include <linux/phy.h> +#include <linux/processor.h> +#include <net/dst_metadata.h> +#include <net/macsec.h> + +#include "nxp-c45-tja11xx.h" + +#define MACSEC_REG_SIZE 32 +#define TX_SC_MAX 4 + +#define TX_SC_BIT(secy_id) BIT(MACSEC_REG_SIZE - (secy_id) - 1) + +#define VEND1_MACSEC_BASE 0x9000 + +#define MACSEC_CFG 0x0000 +#define MACSEC_CFG_BYPASS BIT(1) +#define MACSEC_CFG_S0I BIT(0) + +#define MACSEC_TPNET 0x0044 +#define PN_WRAP_THRESHOLD 0xffffffff + +#define MACSEC_RXSCA 0x0080 +#define MACSEC_RXSCKA 0x0084 + +#define MACSEC_TXSCA 0x00C0 +#define MACSEC_TXSCKA 0x00C4 + +#define MACSEC_RXSC_SCI_1H 0x0100 + +#define MACSEC_RXSC_CFG 0x0128 +#define MACSEC_RXSC_CFG_XPN BIT(25) +#define MACSEC_RXSC_CFG_AES_256 BIT(24) +#define MACSEC_RXSC_CFG_SCI_EN BIT(11) +#define MACSEC_RXSC_CFG_RP BIT(10) +#define MACSEC_RXSC_CFG_VF_MASK GENMASK(9, 8) +#define MACSEC_RXSC_CFG_VF_OFF 8 + +#define MACSEC_RPW 0x012C + +#define MACSEC_RXSA_A_CS 0x0180 +#define MACSEC_RXSA_A_NPN 0x0184 +#define MACSEC_RXSA_A_XNPN 0x0188 +#define MACSEC_RXSA_A_LNPN 0x018C +#define MACSEC_RXSA_A_LXNPN 0x0190 + +#define MACSEC_RXSA_B_CS 0x01C0 +#define MACSEC_RXSA_B_NPN 0x01C4 +#define MACSEC_RXSA_B_XNPN 0x01C8 +#define MACSEC_RXSA_B_LNPN 0x01CC +#define MACSEC_RXSA_B_LXNPN 0x01D0 + +#define MACSEC_RXSA_CS_AN_OFF 1 +#define MACSEC_RXSA_CS_EN BIT(0) + +#define MACSEC_TXSC_SCI_1H 0x0200 +#define MACSEC_TXSC_CFG 0x0228 +#define MACSEC_TXSC_CFG_XPN BIT(25) +#define MACSEC_TXSC_CFG_AES_256 BIT(24) +#define MACSEC_TXSC_CFG_AN_MASK GENMASK(19, 18) +#define MACSEC_TXSC_CFG_AN_OFF 18 +#define MACSEC_TXSC_CFG_ASA BIT(17) +#define MACSEC_TXSC_CFG_SCE BIT(16) +#define MACSEC_TXSC_CFG_ENCRYPT BIT(4) +#define MACSEC_TXSC_CFG_PROTECT BIT(3) +#define MACSEC_TXSC_CFG_SEND_SCI BIT(2) +#define MACSEC_TXSC_CFG_END_STATION BIT(1) +#define MACSEC_TXSC_CFG_SCB BIT(0) + +#define MACSEC_TXSA_A_CS 0x0280 +#define MACSEC_TXSA_A_NPN 0x0284 +#define MACSEC_TXSA_A_XNPN 0x0288 + +#define MACSEC_TXSA_B_CS 0x02C0 +#define MACSEC_TXSA_B_NPN 0x02C4 +#define MACSEC_TXSA_B_XNPN 0x02C8 + +#define MACSEC_SA_CS_A BIT(31) + +#define MACSEC_EVR 0x0400 +#define MACSEC_EVER 0x0404 + +#define MACSEC_RXSA_A_KA 0x0700 +#define MACSEC_RXSA_A_SSCI 0x0720 +#define MACSEC_RXSA_A_SALT 0x0724 + +#define MACSEC_RXSA_B_KA 0x0740 +#define MACSEC_RXSA_B_SSCI 0x0760 +#define MACSEC_RXSA_B_SALT 0x0764 + +#define MACSEC_TXSA_A_KA 0x0780 +#define MACSEC_TXSA_A_SSCI 0x07A0 +#define MACSEC_TXSA_A_SALT 0x07A4 + +#define MACSEC_TXSA_B_KA 0x07C0 +#define MACSEC_TXSA_B_SSCI 0x07E0 +#define MACSEC_TXSA_B_SALT 0x07E4 + +#define MACSEC_UPFR0D2 0x0A08 +#define MACSEC_UPFR0M1 0x0A10 +#define MACSEC_OVP BIT(12) + +#define MACSEC_UPFR0M2 0x0A14 +#define ETYPE_MASK 0xffff + +#define MACSEC_UPFR0R 0x0A18 +#define MACSEC_UPFR_EN BIT(0) + +#define ADPTR_CNTRL 0x0F00 +#define ADPTR_CNTRL_CONFIG_EN BIT(14) +#define ADPTR_CNTRL_ADPTR_EN BIT(12) +#define ADPTR_TX_TAG_CNTRL 0x0F0C +#define ADPTR_TX_TAG_CNTRL_ENA BIT(31) + +#define TX_SC_FLT_BASE 0x800 +#define TX_SC_FLT_SIZE 0x10 +#define TX_FLT_BASE(flt_id) (TX_SC_FLT_BASE + \ + TX_SC_FLT_SIZE * (flt_id)) + +#define TX_SC_FLT_OFF_MAC_DA_SA 0x04 +#define TX_SC_FLT_OFF_MAC_SA 0x08 +#define TX_SC_FLT_OFF_MAC_CFG 0x0C +#define TX_SC_FLT_BY_SA BIT(14) +#define TX_SC_FLT_EN BIT(8) + +#define TX_SC_FLT_MAC_DA_SA(base) ((base) + TX_SC_FLT_OFF_MAC_DA_SA) +#define TX_SC_FLT_MAC_SA(base) ((base) + TX_SC_FLT_OFF_MAC_SA) +#define TX_SC_FLT_MAC_CFG(base) ((base) + TX_SC_FLT_OFF_MAC_CFG) + +#define ADAPTER_EN BIT(6) +#define MACSEC_EN BIT(5) + +#define MACSEC_INOV1HS 0x0140 +#define MACSEC_INOV2HS 0x0144 +#define MACSEC_INOD1HS 0x0148 +#define MACSEC_INOD2HS 0x014C +#define MACSEC_RXSCIPUS 0x0150 +#define MACSEC_RXSCIPDS 0x0154 +#define MACSEC_RXSCIPLS 0x0158 +#define MACSEC_RXAN0INUSS 0x0160 +#define MACSEC_RXAN0IPUSS 0x0170 +#define MACSEC_RXSA_A_IPOS 0x0194 +#define MACSEC_RXSA_A_IPIS 0x01B0 +#define MACSEC_RXSA_A_IPNVS 0x01B4 +#define MACSEC_RXSA_B_IPOS 0x01D4 +#define MACSEC_RXSA_B_IPIS 0x01F0 +#define MACSEC_RXSA_B_IPNVS 0x01F4 +#define MACSEC_OPUS 0x021C +#define MACSEC_OPTLS 0x022C +#define MACSEC_OOP1HS 0x0240 +#define MACSEC_OOP2HS 0x0244 +#define MACSEC_OOE1HS 0x0248 +#define MACSEC_OOE2HS 0x024C +#define MACSEC_TXSA_A_OPPS 0x028C +#define MACSEC_TXSA_A_OPES 0x0290 +#define MACSEC_TXSA_B_OPPS 0x02CC +#define MACSEC_TXSA_B_OPES 0x02D0 +#define MACSEC_INPWTS 0x0630 +#define MACSEC_INPBTS 0x0638 +#define MACSEC_IPSNFS 0x063C + +#define TJA11XX_TLV_TX_NEEDED_HEADROOM (32) +#define TJA11XX_TLV_NEEDED_TAILROOM (0) + +#define ETH_P_TJA11XX_TLV (0x4e58) + +enum nxp_c45_sa_type { + TX_SA, + RX_SA, +}; + +struct nxp_c45_sa { + void *sa; + const struct nxp_c45_sa_regs *regs; + enum nxp_c45_sa_type type; + bool is_key_a; + u8 an; + struct list_head list; +}; + +struct nxp_c45_secy { + struct macsec_secy *secy; + struct macsec_rx_sc *rx_sc; + struct list_head sa_list; + int secy_id; + bool rx_sc0_impl; + struct list_head list; +}; + +struct nxp_c45_macsec { + struct list_head secy_list; + DECLARE_BITMAP(secy_bitmap, TX_SC_MAX); + DECLARE_BITMAP(tx_sc_bitmap, TX_SC_MAX); +}; + +struct nxp_c45_sa_regs { + u16 cs; + u16 npn; + u16 xnpn; + u16 lnpn; + u16 lxnpn; + u16 ka; + u16 ssci; + u16 salt; + u16 ipis; + u16 ipnvs; + u16 ipos; + u16 opps; + u16 opes; +}; + +static const struct nxp_c45_sa_regs rx_sa_a_regs = { + .cs = MACSEC_RXSA_A_CS, + .npn = MACSEC_RXSA_A_NPN, + .xnpn = MACSEC_RXSA_A_XNPN, + .lnpn = MACSEC_RXSA_A_LNPN, + .lxnpn = MACSEC_RXSA_A_LXNPN, + .ka = MACSEC_RXSA_A_KA, + .ssci = MACSEC_RXSA_A_SSCI, + .salt = MACSEC_RXSA_A_SALT, + .ipis = MACSEC_RXSA_A_IPIS, + .ipnvs = MACSEC_RXSA_A_IPNVS, + .ipos = MACSEC_RXSA_A_IPOS, +}; + +static const struct nxp_c45_sa_regs rx_sa_b_regs = { + .cs = MACSEC_RXSA_B_CS, + .npn = MACSEC_RXSA_B_NPN, + .xnpn = MACSEC_RXSA_B_XNPN, + .lnpn = MACSEC_RXSA_B_LNPN, + .lxnpn = MACSEC_RXSA_B_LXNPN, + .ka = MACSEC_RXSA_B_KA, + .ssci = MACSEC_RXSA_B_SSCI, + .salt = MACSEC_RXSA_B_SALT, + .ipis = MACSEC_RXSA_B_IPIS, + .ipnvs = MACSEC_RXSA_B_IPNVS, + .ipos = MACSEC_RXSA_B_IPOS, +}; + +static const struct nxp_c45_sa_regs tx_sa_a_regs = { + .cs = MACSEC_TXSA_A_CS, + .npn = MACSEC_TXSA_A_NPN, + .xnpn = MACSEC_TXSA_A_XNPN, + .ka = MACSEC_TXSA_A_KA, + .ssci = MACSEC_TXSA_A_SSCI, + .salt = MACSEC_TXSA_A_SALT, + .opps = MACSEC_TXSA_A_OPPS, + .opes = MACSEC_TXSA_A_OPES, +}; + +static const struct nxp_c45_sa_regs tx_sa_b_regs = { + .cs = MACSEC_TXSA_B_CS, + .npn = MACSEC_TXSA_B_NPN, + .xnpn = MACSEC_TXSA_B_XNPN, + .ka = MACSEC_TXSA_B_KA, + .ssci = MACSEC_TXSA_B_SSCI, + .salt = MACSEC_TXSA_B_SALT, + .opps = MACSEC_TXSA_B_OPPS, + .opes = MACSEC_TXSA_B_OPES, +}; + +static const +struct nxp_c45_sa_regs *nxp_c45_sa_regs_get(enum nxp_c45_sa_type sa_type, + bool key_a) +{ + if (sa_type == RX_SA) + if (key_a) + return &rx_sa_a_regs; + else + return &rx_sa_b_regs; + else if (sa_type == TX_SA) + if (key_a) + return &tx_sa_a_regs; + else + return &tx_sa_b_regs; + else + return NULL; +} + +static int nxp_c45_macsec_write(struct phy_device *phydev, u16 addr, u32 value) +{ + u32 lvalue = value; + u16 laddr; + int ret; + + WARN_ON_ONCE(addr % 4); + + phydev_dbg(phydev, "write addr 0x%x value 0x%x\n", addr, value); + + laddr = VEND1_MACSEC_BASE + addr / 2; + ret = phy_write_mmd(phydev, MDIO_MMD_VEND2, laddr, lvalue); + if (ret) + return ret; + + laddr += 1; + lvalue >>= 16; + ret = phy_write_mmd(phydev, MDIO_MMD_VEND2, laddr, lvalue); + + return ret; +} + +static int nxp_c45_macsec_read(struct phy_device *phydev, u16 addr, u32 *value) +{ + u32 lvalue; + u16 laddr; + int ret; + + WARN_ON_ONCE(addr % 4); + + laddr = VEND1_MACSEC_BASE + addr / 2; + ret = phy_read_mmd(phydev, MDIO_MMD_VEND2, laddr); + if (ret < 0) + return ret; + + laddr += 1; + lvalue = (u32)ret & 0xffff; + ret = phy_read_mmd(phydev, MDIO_MMD_VEND2, laddr); + if (ret < 0) + return ret; + + lvalue |= (u32)ret << 16; + *value = lvalue; + + phydev_dbg(phydev, "read addr 0x%x value 0x%x\n", addr, *value); + + return 0; +} + +static void nxp_c45_macsec_read32_64(struct phy_device *phydev, u16 addr, + u64 *value) +{ + u32 lvalue; + + nxp_c45_macsec_read(phydev, addr, &lvalue); + *value = lvalue; +} + +static void nxp_c45_macsec_read64(struct phy_device *phydev, u16 addr, + u64 *value) +{ + u32 lvalue; + + nxp_c45_macsec_read(phydev, addr, &lvalue); + *value = (u64)lvalue << 32; + nxp_c45_macsec_read(phydev, addr + 4, &lvalue); + *value |= lvalue; +} + +static void nxp_c45_secy_irq_en(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy, bool en) +{ + u32 reg; + + nxp_c45_macsec_read(phydev, MACSEC_EVER, ®); + if (en) + reg |= TX_SC_BIT(phy_secy->secy_id); + else + reg &= ~TX_SC_BIT(phy_secy->secy_id); + nxp_c45_macsec_write(phydev, MACSEC_EVER, reg); +} + +static struct nxp_c45_secy *nxp_c45_find_secy(struct list_head *secy_list, + sci_t sci) +{ + struct nxp_c45_secy *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, secy_list, list) + if (pos->secy->sci == sci) + return pos; + + return ERR_PTR(-EINVAL); +} + +static struct +nxp_c45_secy *nxp_c45_find_secy_by_id(struct list_head *secy_list, + int id) +{ + struct nxp_c45_secy *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, secy_list, list) + if (pos->secy_id == id) + return pos; + + return ERR_PTR(-EINVAL); +} + +static void nxp_c45_secy_free(struct nxp_c45_secy *phy_secy) +{ + list_del(&phy_secy->list); + kfree(phy_secy); +} + +static struct nxp_c45_sa *nxp_c45_find_sa(struct list_head *sa_list, + enum nxp_c45_sa_type sa_type, u8 an) +{ + struct nxp_c45_sa *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, sa_list, list) + if (pos->an == an && pos->type == sa_type) + return pos; + + return ERR_PTR(-EINVAL); +} + +static struct nxp_c45_sa *nxp_c45_sa_alloc(struct list_head *sa_list, void *sa, + enum nxp_c45_sa_type sa_type, u8 an) +{ + struct nxp_c45_sa *first = NULL, *pos, *tmp; + int occurrences = 0; + + list_for_each_entry_safe(pos, tmp, sa_list, list) { + if (pos->type != sa_type) + continue; + + if (pos->an == an) + return ERR_PTR(-EINVAL); + + first = pos; + occurrences++; + if (occurrences >= 2) + return ERR_PTR(-ENOSPC); + } + + tmp = kzalloc(sizeof(*tmp), GFP_KERNEL); + if (!tmp) + return ERR_PTR(-ENOMEM); + + if (first) + tmp->is_key_a = !first->is_key_a; + else + tmp->is_key_a = true; + + tmp->sa = sa; + tmp->type = sa_type; + tmp->an = an; + tmp->regs = nxp_c45_sa_regs_get(tmp->type, tmp->is_key_a); + list_add_tail(&tmp->list, sa_list); + + return tmp; +} + +static void nxp_c45_sa_free(struct nxp_c45_sa *sa) +{ + list_del(&sa->list); + kfree(sa); +} + +static void nxp_c45_sa_list_free(struct list_head *sa_list) +{ + struct nxp_c45_sa *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, sa_list, list) + nxp_c45_sa_free(pos); +} + +static void nxp_c45_sa_set_pn(struct phy_device *phydev, + struct nxp_c45_sa *sa, u64 pn, + u32 replay_window) +{ + const struct nxp_c45_sa_regs *sa_regs = sa->regs; + pn_t npn = {.full64 = pn}; + pn_t lnpn; + + nxp_c45_macsec_write(phydev, sa_regs->npn, npn.lower); + nxp_c45_macsec_write(phydev, sa_regs->xnpn, npn.upper); + if (sa->type != RX_SA) + return; + + if (pn > replay_window) + lnpn.full64 = pn - replay_window; + else + lnpn.full64 = 1; + + nxp_c45_macsec_write(phydev, sa_regs->lnpn, lnpn.lower); + nxp_c45_macsec_write(phydev, sa_regs->lxnpn, lnpn.upper); +} + +static void nxp_c45_sa_set_key(struct macsec_context *ctx, + const struct nxp_c45_sa_regs *sa_regs, + u8 *salt, ssci_t ssci) +{ + struct phy_device *phydev = ctx->phydev; + u32 key_size = ctx->secy->key_len / 4; + u32 salt_size = MACSEC_SALT_LEN / 4; + u32 *key_u32 = (u32 *)ctx->sa.key; + u32 *salt_u32 = (u32 *)salt; + u32 reg, value; + int i; + + for (i = 0; i < key_size; i++) { + reg = sa_regs->ka + i * 4; + value = (__force u32)cpu_to_be32(key_u32[i]); + nxp_c45_macsec_write(phydev, reg, value); + } + + if (ctx->secy->xpn) { + for (i = 0; i < salt_size; i++) { + reg = sa_regs->salt + (2 - i) * 4; + value = (__force u32)cpu_to_be32(salt_u32[i]); + nxp_c45_macsec_write(phydev, reg, value); + } + + value = (__force u32)cpu_to_be32((__force u32)ssci); + nxp_c45_macsec_write(phydev, sa_regs->ssci, value); + } + + nxp_c45_macsec_write(phydev, sa_regs->cs, MACSEC_SA_CS_A); +} + +static void nxp_c45_rx_sa_clear_stats(struct phy_device *phydev, + struct nxp_c45_sa *sa) +{ + nxp_c45_macsec_write(phydev, sa->regs->ipis, 0); + nxp_c45_macsec_write(phydev, sa->regs->ipnvs, 0); + nxp_c45_macsec_write(phydev, sa->regs->ipos, 0); + + nxp_c45_macsec_write(phydev, MACSEC_RXAN0INUSS + sa->an * 4, 0); + nxp_c45_macsec_write(phydev, MACSEC_RXAN0IPUSS + sa->an * 4, 0); +} + +static void nxp_c45_rx_sa_read_stats(struct phy_device *phydev, + struct nxp_c45_sa *sa, + struct macsec_rx_sa_stats *stats) +{ + nxp_c45_macsec_read(phydev, sa->regs->ipis, &stats->InPktsInvalid); + nxp_c45_macsec_read(phydev, sa->regs->ipnvs, &stats->InPktsNotValid); + nxp_c45_macsec_read(phydev, sa->regs->ipos, &stats->InPktsOK); +} + +static void nxp_c45_tx_sa_clear_stats(struct phy_device *phydev, + struct nxp_c45_sa *sa) +{ + nxp_c45_macsec_write(phydev, sa->regs->opps, 0); + nxp_c45_macsec_write(phydev, sa->regs->opes, 0); +} + +static void nxp_c45_tx_sa_read_stats(struct phy_device *phydev, + struct nxp_c45_sa *sa, + struct macsec_tx_sa_stats *stats) +{ + nxp_c45_macsec_read(phydev, sa->regs->opps, &stats->OutPktsProtected); + nxp_c45_macsec_read(phydev, sa->regs->opes, &stats->OutPktsEncrypted); +} + +static void nxp_c45_rx_sa_update(struct phy_device *phydev, + struct nxp_c45_sa *sa, bool en) +{ + const struct nxp_c45_sa_regs *sa_regs = sa->regs; + u32 cfg; + + cfg = sa->an << MACSEC_RXSA_CS_AN_OFF; + cfg |= en ? MACSEC_RXSA_CS_EN : 0; + nxp_c45_macsec_write(phydev, sa_regs->cs, cfg); +} + +static void nxp_c45_tx_sa_update(struct phy_device *phydev, + struct nxp_c45_sa *sa, bool en) +{ + u32 cfg = 0; + + nxp_c45_macsec_read(phydev, MACSEC_TXSC_CFG, &cfg); + + cfg &= ~MACSEC_TXSC_CFG_AN_MASK; + cfg |= sa->an << MACSEC_TXSC_CFG_AN_OFF; + + if (sa->is_key_a) + cfg &= ~MACSEC_TXSC_CFG_ASA; + else + cfg |= MACSEC_TXSC_CFG_ASA; + + if (en) + cfg |= MACSEC_TXSC_CFG_SCE; + else + cfg &= ~MACSEC_TXSC_CFG_SCE; + + nxp_c45_macsec_write(phydev, MACSEC_TXSC_CFG, cfg); +} + +static void nxp_c45_set_sci(struct phy_device *phydev, u16 sci_base_addr, + sci_t sci) +{ + u64 lsci = sci_to_cpu(sci); + + nxp_c45_macsec_write(phydev, sci_base_addr, lsci >> 32); + nxp_c45_macsec_write(phydev, sci_base_addr + 4, lsci); +} + +static bool nxp_c45_port_is_1(sci_t sci) +{ + u16 port = sci_to_cpu(sci); + + return port == 1; +} + +static void nxp_c45_select_secy(struct phy_device *phydev, u8 id) +{ + nxp_c45_macsec_write(phydev, MACSEC_RXSCA, id); + nxp_c45_macsec_write(phydev, MACSEC_RXSCKA, id); + nxp_c45_macsec_write(phydev, MACSEC_TXSCA, id); + nxp_c45_macsec_write(phydev, MACSEC_TXSCKA, id); +} + +static bool nxp_c45_secy_valid(struct nxp_c45_secy *phy_secy, + bool can_rx_sc0_impl) +{ + bool end_station = phy_secy->secy->tx_sc.end_station; + bool scb = phy_secy->secy->tx_sc.scb; + + phy_secy->rx_sc0_impl = false; + + if (end_station) { + if (!nxp_c45_port_is_1(phy_secy->secy->sci)) + return false; + if (!phy_secy->rx_sc) + return true; + return nxp_c45_port_is_1(phy_secy->rx_sc->sci); + } + + if (scb) + return false; + + if (!can_rx_sc0_impl) + return false; + + if (phy_secy->secy_id != 0) + return false; + + phy_secy->rx_sc0_impl = true; + + return true; +} + +static bool nxp_c45_rx_sc0_impl(struct nxp_c45_secy *phy_secy) +{ + bool end_station = phy_secy->secy->tx_sc.end_station; + bool send_sci = phy_secy->secy->tx_sc.send_sci; + bool scb = phy_secy->secy->tx_sc.scb; + + return !end_station && !send_sci && !scb; +} + +static bool nxp_c45_mac_addr_free(struct macsec_context *ctx) +{ + struct nxp_c45_phy *priv = ctx->phydev->priv; + struct nxp_c45_secy *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, &priv->macsec->secy_list, list) { + if (pos->secy == ctx->secy) + continue; + + if (memcmp(pos->secy->netdev->dev_addr, + ctx->secy->netdev->dev_addr, ETH_ALEN) == 0) + return false; + } + + return true; +} + +static void nxp_c45_tx_sc_en_flt(struct phy_device *phydev, int secy_id, + bool en) +{ + u32 tx_flt_base = TX_FLT_BASE(secy_id); + u32 reg = 0; + + nxp_c45_macsec_read(phydev, TX_SC_FLT_MAC_CFG(tx_flt_base), ®); + if (en) + reg |= TX_SC_FLT_EN; + else + reg &= ~TX_SC_FLT_EN; + nxp_c45_macsec_write(phydev, TX_SC_FLT_MAC_CFG(tx_flt_base), reg); +} + +static void nxp_c45_tx_sc_set_flt(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + const u8 *dev_addr = phy_secy->secy->netdev->dev_addr; + u32 tx_flt_base = TX_FLT_BASE(phy_secy->secy_id); + u32 reg; + + reg = dev_addr[0] << 8 | dev_addr[1]; + nxp_c45_macsec_write(phydev, TX_SC_FLT_MAC_DA_SA(tx_flt_base), reg); + reg = dev_addr[5] | dev_addr[4] << 8 | dev_addr[3] << 16 | + dev_addr[2] << 24; + + nxp_c45_macsec_write(phydev, TX_SC_FLT_MAC_SA(tx_flt_base), reg); + nxp_c45_macsec_read(phydev, TX_SC_FLT_MAC_CFG(tx_flt_base), ®); + reg &= TX_SC_FLT_EN; + reg |= TX_SC_FLT_BY_SA | phy_secy->secy_id; + nxp_c45_macsec_write(phydev, TX_SC_FLT_MAC_CFG(tx_flt_base), reg); +} + +static void nxp_c45_tx_sc_update(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + u32 cfg = 0; + + nxp_c45_macsec_read(phydev, MACSEC_TXSC_CFG, &cfg); + + phydev_dbg(phydev, "XPN %s\n", phy_secy->secy->xpn ? "on" : "off"); + if (phy_secy->secy->xpn) + cfg |= MACSEC_TXSC_CFG_XPN; + else + cfg &= ~MACSEC_TXSC_CFG_XPN; + + phydev_dbg(phydev, "key len %u\n", phy_secy->secy->key_len); + if (phy_secy->secy->key_len == 32) + cfg |= MACSEC_TXSC_CFG_AES_256; + else + cfg &= ~MACSEC_TXSC_CFG_AES_256; + + phydev_dbg(phydev, "encryption %s\n", + phy_secy->secy->tx_sc.encrypt ? "on" : "off"); + if (phy_secy->secy->tx_sc.encrypt) + cfg |= MACSEC_TXSC_CFG_ENCRYPT; + else + cfg &= ~MACSEC_TXSC_CFG_ENCRYPT; + + phydev_dbg(phydev, "protect frames %s\n", + phy_secy->secy->protect_frames ? "on" : "off"); + if (phy_secy->secy->protect_frames) + cfg |= MACSEC_TXSC_CFG_PROTECT; + else + cfg &= ~MACSEC_TXSC_CFG_PROTECT; + + phydev_dbg(phydev, "send sci %s\n", + phy_secy->secy->tx_sc.send_sci ? "on" : "off"); + if (phy_secy->secy->tx_sc.send_sci) + cfg |= MACSEC_TXSC_CFG_SEND_SCI; + else + cfg &= ~MACSEC_TXSC_CFG_SEND_SCI; + + phydev_dbg(phydev, "end station %s\n", + phy_secy->secy->tx_sc.end_station ? "on" : "off"); + if (phy_secy->secy->tx_sc.end_station) + cfg |= MACSEC_TXSC_CFG_END_STATION; + else + cfg &= ~MACSEC_TXSC_CFG_END_STATION; + + phydev_dbg(phydev, "scb %s\n", + phy_secy->secy->tx_sc.scb ? "on" : "off"); + if (phy_secy->secy->tx_sc.scb) + cfg |= MACSEC_TXSC_CFG_SCB; + else + cfg &= ~MACSEC_TXSC_CFG_SCB; + + nxp_c45_macsec_write(phydev, MACSEC_TXSC_CFG, cfg); +} + +static void nxp_c45_tx_sc_clear_stats(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + struct nxp_c45_sa *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, &phy_secy->sa_list, list) + if (pos->type == TX_SA) + nxp_c45_tx_sa_clear_stats(phydev, pos); + + nxp_c45_macsec_write(phydev, MACSEC_OPUS, 0); + nxp_c45_macsec_write(phydev, MACSEC_OPTLS, 0); + nxp_c45_macsec_write(phydev, MACSEC_OOP1HS, 0); + nxp_c45_macsec_write(phydev, MACSEC_OOP2HS, 0); + nxp_c45_macsec_write(phydev, MACSEC_OOE1HS, 0); + nxp_c45_macsec_write(phydev, MACSEC_OOE2HS, 0); +} + +static void nxp_c45_set_rx_sc0_impl(struct phy_device *phydev, + bool enable) +{ + u32 reg = 0; + + nxp_c45_macsec_read(phydev, MACSEC_CFG, ®); + if (enable) + reg |= MACSEC_CFG_S0I; + else + reg &= ~MACSEC_CFG_S0I; + nxp_c45_macsec_write(phydev, MACSEC_CFG, reg); +} + +static bool nxp_c45_is_rx_sc0_impl(struct list_head *secy_list) +{ + struct nxp_c45_secy *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, secy_list, list) + if (pos->rx_sc0_impl) + return pos->rx_sc0_impl; + + return false; +} + +static void nxp_c45_rx_sc_en(struct phy_device *phydev, + struct macsec_rx_sc *rx_sc, bool en) +{ + u32 reg = 0; + + nxp_c45_macsec_read(phydev, MACSEC_RXSC_CFG, ®); + if (rx_sc->active && en) + reg |= MACSEC_RXSC_CFG_SCI_EN; + else + reg &= ~MACSEC_RXSC_CFG_SCI_EN; + nxp_c45_macsec_write(phydev, MACSEC_RXSC_CFG, reg); +} + +static void nxp_c45_rx_sc_update(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + struct macsec_rx_sc *rx_sc = phy_secy->rx_sc; + struct nxp_c45_phy *priv = phydev->priv; + u32 cfg = 0; + + nxp_c45_macsec_read(phydev, MACSEC_RXSC_CFG, &cfg); + cfg &= ~MACSEC_RXSC_CFG_VF_MASK; + cfg = phy_secy->secy->validate_frames << MACSEC_RXSC_CFG_VF_OFF; + + phydev_dbg(phydev, "validate frames %u\n", + phy_secy->secy->validate_frames); + phydev_dbg(phydev, "replay_protect %s window %u\n", + phy_secy->secy->replay_protect ? "on" : "off", + phy_secy->secy->replay_window); + if (phy_secy->secy->replay_protect) { + cfg |= MACSEC_RXSC_CFG_RP; + nxp_c45_macsec_write(phydev, MACSEC_RPW, + phy_secy->secy->replay_window); + } else { + cfg &= ~MACSEC_RXSC_CFG_RP; + } + + phydev_dbg(phydev, "rx_sc->active %s\n", + rx_sc->active ? "on" : "off"); + if (rx_sc->active && + test_bit(phy_secy->secy_id, priv->macsec->secy_bitmap)) + cfg |= MACSEC_RXSC_CFG_SCI_EN; + else + cfg &= ~MACSEC_RXSC_CFG_SCI_EN; + + phydev_dbg(phydev, "key len %u\n", phy_secy->secy->key_len); + if (phy_secy->secy->key_len == 32) + cfg |= MACSEC_RXSC_CFG_AES_256; + else + cfg &= ~MACSEC_RXSC_CFG_AES_256; + + phydev_dbg(phydev, "XPN %s\n", phy_secy->secy->xpn ? "on" : "off"); + if (phy_secy->secy->xpn) + cfg |= MACSEC_RXSC_CFG_XPN; + else + cfg &= ~MACSEC_RXSC_CFG_XPN; + + nxp_c45_macsec_write(phydev, MACSEC_RXSC_CFG, cfg); +} + +static void nxp_c45_rx_sc_clear_stats(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + struct nxp_c45_sa *pos, *tmp; + int i; + + list_for_each_entry_safe(pos, tmp, &phy_secy->sa_list, list) + if (pos->type == RX_SA) + nxp_c45_rx_sa_clear_stats(phydev, pos); + + nxp_c45_macsec_write(phydev, MACSEC_INOD1HS, 0); + nxp_c45_macsec_write(phydev, MACSEC_INOD2HS, 0); + + nxp_c45_macsec_write(phydev, MACSEC_INOV1HS, 0); + nxp_c45_macsec_write(phydev, MACSEC_INOV2HS, 0); + + nxp_c45_macsec_write(phydev, MACSEC_RXSCIPDS, 0); + nxp_c45_macsec_write(phydev, MACSEC_RXSCIPLS, 0); + nxp_c45_macsec_write(phydev, MACSEC_RXSCIPUS, 0); + + for (i = 0; i < MACSEC_NUM_AN; i++) { + nxp_c45_macsec_write(phydev, MACSEC_RXAN0INUSS + i * 4, 0); + nxp_c45_macsec_write(phydev, MACSEC_RXAN0IPUSS + i * 4, 0); + } +} + +static void nxp_c45_rx_sc_del(struct phy_device *phydev, + struct nxp_c45_secy *phy_secy) +{ + struct nxp_c45_sa *pos, *tmp; + + nxp_c45_macsec_write(phydev, MACSEC_RXSC_CFG, 0); + nxp_c45_macsec_write(phydev, MACSEC_RPW, 0); + nxp_c45_set_sci(phydev, MACSEC_RXSC_SCI_1H, 0); + + nxp_c45_rx_sc_clear_stats(phydev, phy_secy); + + list_for_each_entry_safe(pos, tmp, &phy_secy->sa_list, list) { + if (pos->type == RX_SA) { + nxp_c45_rx_sa_update(phydev, pos, false); + nxp_c45_sa_free(pos); + } + } +} + +static void nxp_c45_clear_global_stats(struct phy_device *phydev) +{ + nxp_c45_macsec_write(phydev, MACSEC_INPBTS, 0); + nxp_c45_macsec_write(phydev, MACSEC_INPWTS, 0); + nxp_c45_macsec_write(phydev, MACSEC_IPSNFS, 0); +} + +static void nxp_c45_macsec_en(struct phy_device *phydev, bool en) +{ + u32 reg; + + nxp_c45_macsec_read(phydev, MACSEC_CFG, ®); + if (en) + reg |= MACSEC_CFG_BYPASS; + else + reg &= ~MACSEC_CFG_BYPASS; + nxp_c45_macsec_write(phydev, MACSEC_CFG, reg); +} + +static int nxp_c45_mdo_dev_open(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + int any_bit_set; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_tx_sc_en_flt(phydev, phy_secy->secy_id, true); + nxp_c45_set_rx_sc0_impl(phydev, phy_secy->rx_sc0_impl); + if (phy_secy->rx_sc) + nxp_c45_rx_sc_en(phydev, phy_secy->rx_sc, true); + + any_bit_set = find_first_bit(priv->macsec->secy_bitmap, TX_SC_MAX); + if (any_bit_set == TX_SC_MAX) + nxp_c45_macsec_en(phydev, true); + + set_bit(phy_secy->secy_id, priv->macsec->secy_bitmap); + + return 0; +} + +static int nxp_c45_mdo_dev_stop(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + int any_bit_set; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_tx_sc_en_flt(phydev, phy_secy->secy_id, false); + if (phy_secy->rx_sc) + nxp_c45_rx_sc_en(phydev, phy_secy->rx_sc, false); + nxp_c45_set_rx_sc0_impl(phydev, false); + + clear_bit(phy_secy->secy_id, priv->macsec->secy_bitmap); + any_bit_set = find_first_bit(priv->macsec->secy_bitmap, TX_SC_MAX); + if (any_bit_set == TX_SC_MAX) + nxp_c45_macsec_en(phydev, false); + + return 0; +} + +static int nxp_c45_mdo_add_secy(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + bool can_rx_sc0_impl; + int idx; + + phydev_dbg(phydev, "add SecY SCI %016llx\n", + sci_to_cpu(ctx->secy->sci)); + + if (!nxp_c45_mac_addr_free(ctx)) + return -EBUSY; + + if (nxp_c45_is_rx_sc0_impl(&priv->macsec->secy_list)) + return -EBUSY; + + idx = find_first_zero_bit(priv->macsec->tx_sc_bitmap, TX_SC_MAX); + if (idx == TX_SC_MAX) + return -ENOSPC; + + phy_secy = kzalloc(sizeof(*phy_secy), GFP_KERNEL); + if (!phy_secy) + return -ENOMEM; + + INIT_LIST_HEAD(&phy_secy->sa_list); + phy_secy->secy = ctx->secy; + phy_secy->secy_id = idx; + + /* If the point to point mode should be enabled, we should have no + * SecY added yet. + */ + can_rx_sc0_impl = list_count_nodes(&priv->macsec->secy_list) == 0; + if (!nxp_c45_secy_valid(phy_secy, can_rx_sc0_impl)) { + kfree(phy_secy); + return -EINVAL; + } + + phy_secy->rx_sc0_impl = nxp_c45_rx_sc0_impl(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_set_sci(phydev, MACSEC_TXSC_SCI_1H, ctx->secy->sci); + nxp_c45_tx_sc_set_flt(phydev, phy_secy); + nxp_c45_tx_sc_update(phydev, phy_secy); + if (phy_interrupt_is_valid(phydev)) + nxp_c45_secy_irq_en(phydev, phy_secy, true); + + set_bit(idx, priv->macsec->tx_sc_bitmap); + list_add_tail(&phy_secy->list, &priv->macsec->secy_list); + + return 0; +} + +static void nxp_c45_tx_sa_next(struct nxp_c45_secy *phy_secy, + struct nxp_c45_sa *next_sa, u8 encoding_sa) +{ + struct nxp_c45_sa *sa; + + sa = nxp_c45_find_sa(&phy_secy->sa_list, TX_SA, encoding_sa); + if (!IS_ERR(sa)) { + memcpy(next_sa, sa, sizeof(*sa)); + } else { + next_sa->is_key_a = true; + next_sa->an = encoding_sa; + } +} + +static int nxp_c45_mdo_upd_secy(struct macsec_context *ctx) +{ + u8 encoding_sa = ctx->secy->tx_sc.encoding_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + struct nxp_c45_sa next_sa; + bool can_rx_sc0_impl; + + phydev_dbg(phydev, "update SecY SCI %016llx\n", + sci_to_cpu(ctx->secy->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + if (!nxp_c45_mac_addr_free(ctx)) + return -EBUSY; + + /* If the point to point mode should be enabled, we should have only + * one SecY added, respectively the updated one. + */ + can_rx_sc0_impl = list_count_nodes(&priv->macsec->secy_list) == 1; + if (!nxp_c45_secy_valid(phy_secy, can_rx_sc0_impl)) + return -EINVAL; + phy_secy->rx_sc0_impl = nxp_c45_rx_sc0_impl(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_tx_sc_set_flt(phydev, phy_secy); + nxp_c45_tx_sc_update(phydev, phy_secy); + nxp_c45_tx_sa_next(phy_secy, &next_sa, encoding_sa); + nxp_c45_tx_sa_update(phydev, &next_sa, ctx->secy->operational); + + nxp_c45_set_rx_sc0_impl(phydev, phy_secy->rx_sc0_impl); + if (phy_secy->rx_sc) + nxp_c45_rx_sc_update(phydev, phy_secy); + + return 0; +} + +static int nxp_c45_mdo_del_secy(struct macsec_context *ctx) +{ + u8 encoding_sa = ctx->secy->tx_sc.encoding_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + struct nxp_c45_sa next_sa; + + phydev_dbg(phydev, "delete SecY SCI %016llx\n", + sci_to_cpu(ctx->secy->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_mdo_dev_stop(ctx); + nxp_c45_tx_sa_next(phy_secy, &next_sa, encoding_sa); + nxp_c45_tx_sa_update(phydev, &next_sa, false); + nxp_c45_tx_sc_clear_stats(phydev, phy_secy); + if (phy_secy->rx_sc) + nxp_c45_rx_sc_del(phydev, phy_secy); + + nxp_c45_sa_list_free(&phy_secy->sa_list); + if (phy_interrupt_is_valid(phydev)) + nxp_c45_secy_irq_en(phydev, phy_secy, false); + + clear_bit(phy_secy->secy_id, priv->macsec->tx_sc_bitmap); + nxp_c45_secy_free(phy_secy); + + if (list_empty(&priv->macsec->secy_list)) + nxp_c45_clear_global_stats(phydev); + + return 0; +} + +static int nxp_c45_mdo_add_rxsc(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + + phydev_dbg(phydev, "add RX SC SCI %016llx %s\n", + sci_to_cpu(ctx->rx_sc->sci), + ctx->rx_sc->active ? "enabled" : "disabled"); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + if (phy_secy->rx_sc) + return -ENOSPC; + + if (phy_secy->secy->tx_sc.end_station && + !nxp_c45_port_is_1(ctx->rx_sc->sci)) + return -EINVAL; + + phy_secy->rx_sc = ctx->rx_sc; + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_set_sci(phydev, MACSEC_RXSC_SCI_1H, ctx->rx_sc->sci); + nxp_c45_rx_sc_update(phydev, phy_secy); + + return 0; +} + +static int nxp_c45_mdo_upd_rxsc(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + + phydev_dbg(phydev, "update RX SC SCI %016llx %s\n", + sci_to_cpu(ctx->rx_sc->sci), + ctx->rx_sc->active ? "enabled" : "disabled"); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_rx_sc_update(phydev, phy_secy); + + return 0; +} + +static int nxp_c45_mdo_del_rxsc(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + + phydev_dbg(phydev, "delete RX SC SCI %016llx %s\n", + sci_to_cpu(ctx->rx_sc->sci), + ctx->rx_sc->active ? "enabled" : "disabled"); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_rx_sc_del(phydev, phy_secy); + phy_secy->rx_sc = NULL; + + return 0; +} + +static int nxp_c45_mdo_add_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sa *rx_sa = ctx->sa.rx_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "add RX SA %u %s to RX SC SCI %016llx\n", + an, rx_sa->active ? "enabled" : "disabled", + sci_to_cpu(rx_sa->sc->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_sa_alloc(&phy_secy->sa_list, rx_sa, RX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_sa_set_pn(phydev, sa, rx_sa->next_pn, + ctx->secy->replay_window); + nxp_c45_sa_set_key(ctx, sa->regs, rx_sa->key.salt.bytes, rx_sa->ssci); + nxp_c45_rx_sa_update(phydev, sa, rx_sa->active); + + return 0; +} + +static int nxp_c45_mdo_upd_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sa *rx_sa = ctx->sa.rx_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "update RX SA %u %s to RX SC SCI %016llx\n", + an, rx_sa->active ? "enabled" : "disabled", + sci_to_cpu(rx_sa->sc->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, RX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + if (ctx->sa.update_pn) + nxp_c45_sa_set_pn(phydev, sa, rx_sa->next_pn, + ctx->secy->replay_window); + nxp_c45_rx_sa_update(phydev, sa, rx_sa->active); + + return 0; +} + +static int nxp_c45_mdo_del_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sa *rx_sa = ctx->sa.rx_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "delete RX SA %u %s to RX SC SCI %016llx\n", + an, rx_sa->active ? "enabled" : "disabled", + sci_to_cpu(rx_sa->sc->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, RX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_rx_sa_update(phydev, sa, false); + nxp_c45_rx_sa_clear_stats(phydev, sa); + + nxp_c45_sa_free(sa); + + return 0; +} + +static int nxp_c45_mdo_add_txsa(struct macsec_context *ctx) +{ + struct macsec_tx_sa *tx_sa = ctx->sa.tx_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "add TX SA %u %s to TX SC %016llx\n", + an, ctx->sa.tx_sa->active ? "enabled" : "disabled", + sci_to_cpu(ctx->secy->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_sa_alloc(&phy_secy->sa_list, tx_sa, TX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_sa_set_pn(phydev, sa, tx_sa->next_pn, 0); + nxp_c45_sa_set_key(ctx, sa->regs, tx_sa->key.salt.bytes, tx_sa->ssci); + if (ctx->secy->tx_sc.encoding_sa == sa->an) + nxp_c45_tx_sa_update(phydev, sa, tx_sa->active); + + return 0; +} + +static int nxp_c45_mdo_upd_txsa(struct macsec_context *ctx) +{ + struct macsec_tx_sa *tx_sa = ctx->sa.tx_sa; + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "update TX SA %u %s to TX SC %016llx\n", + an, ctx->sa.tx_sa->active ? "enabled" : "disabled", + sci_to_cpu(ctx->secy->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, TX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + if (ctx->sa.update_pn) + nxp_c45_sa_set_pn(phydev, sa, tx_sa->next_pn, 0); + if (ctx->secy->tx_sc.encoding_sa == sa->an) + nxp_c45_tx_sa_update(phydev, sa, tx_sa->active); + + return 0; +} + +static int nxp_c45_mdo_del_txsa(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phydev_dbg(phydev, "delete TX SA %u %s to TX SC %016llx\n", + an, ctx->sa.tx_sa->active ? "enabled" : "disabled", + sci_to_cpu(ctx->secy->sci)); + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, TX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + nxp_c45_select_secy(phydev, phy_secy->secy_id); + if (ctx->secy->tx_sc.encoding_sa == sa->an) + nxp_c45_tx_sa_update(phydev, sa, false); + nxp_c45_tx_sa_clear_stats(phydev, sa); + + nxp_c45_sa_free(sa); + + return 0; +} + +static int nxp_c45_mdo_get_dev_stats(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct macsec_dev_stats *dev_stats; + struct nxp_c45_secy *phy_secy; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + dev_stats = ctx->stats.dev_stats; + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_macsec_read32_64(phydev, MACSEC_OPUS, + &dev_stats->OutPktsUntagged); + nxp_c45_macsec_read32_64(phydev, MACSEC_OPTLS, + &dev_stats->OutPktsTooLong); + nxp_c45_macsec_read32_64(phydev, MACSEC_INPBTS, + &dev_stats->InPktsBadTag); + + if (phy_secy->secy->validate_frames == MACSEC_VALIDATE_STRICT) + nxp_c45_macsec_read32_64(phydev, MACSEC_INPWTS, + &dev_stats->InPktsNoTag); + else + nxp_c45_macsec_read32_64(phydev, MACSEC_INPWTS, + &dev_stats->InPktsUntagged); + + if (phy_secy->secy->validate_frames == MACSEC_VALIDATE_STRICT) + nxp_c45_macsec_read32_64(phydev, MACSEC_IPSNFS, + &dev_stats->InPktsNoSCI); + else + nxp_c45_macsec_read32_64(phydev, MACSEC_IPSNFS, + &dev_stats->InPktsUnknownSCI); + + /* Always 0. */ + dev_stats->InPktsOverrun = 0; + + return 0; +} + +static int nxp_c45_mdo_get_tx_sc_stats(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct macsec_tx_sa_stats tx_sa_stats; + struct macsec_tx_sc_stats *stats; + struct nxp_c45_secy *phy_secy; + struct nxp_c45_sa *pos, *tmp; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + stats = ctx->stats.tx_sc_stats; + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_macsec_read64(phydev, MACSEC_OOE1HS, + &stats->OutOctetsEncrypted); + nxp_c45_macsec_read64(phydev, MACSEC_OOP1HS, + &stats->OutOctetsProtected); + list_for_each_entry_safe(pos, tmp, &phy_secy->sa_list, list) { + if (pos->type != TX_SA) + continue; + + memset(&tx_sa_stats, 0, sizeof(tx_sa_stats)); + nxp_c45_tx_sa_read_stats(phydev, pos, &tx_sa_stats); + + stats->OutPktsEncrypted += tx_sa_stats.OutPktsEncrypted; + stats->OutPktsProtected += tx_sa_stats.OutPktsProtected; + } + + return 0; +} + +static int nxp_c45_mdo_get_tx_sa_stats(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct macsec_tx_sa_stats *stats; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, TX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + stats = ctx->stats.tx_sa_stats; + nxp_c45_select_secy(phydev, phy_secy->secy_id); + nxp_c45_tx_sa_read_stats(phydev, sa, stats); + + return 0; +} + +static int nxp_c45_mdo_get_rx_sc_stats(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct macsec_rx_sa_stats rx_sa_stats; + struct macsec_rx_sc_stats *stats; + struct nxp_c45_secy *phy_secy; + struct nxp_c45_sa *pos, *tmp; + u32 reg = 0; + int i; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + if (phy_secy->rx_sc != ctx->rx_sc) + return -EINVAL; + + stats = ctx->stats.rx_sc_stats; + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + list_for_each_entry_safe(pos, tmp, &phy_secy->sa_list, list) { + if (pos->type != RX_SA) + continue; + + memset(&rx_sa_stats, 0, sizeof(rx_sa_stats)); + nxp_c45_rx_sa_read_stats(phydev, pos, &rx_sa_stats); + + stats->InPktsInvalid += rx_sa_stats.InPktsInvalid; + stats->InPktsNotValid += rx_sa_stats.InPktsNotValid; + stats->InPktsOK += rx_sa_stats.InPktsOK; + } + + for (i = 0; i < MACSEC_NUM_AN; i++) { + nxp_c45_macsec_read(phydev, MACSEC_RXAN0INUSS + i * 4, ®); + stats->InPktsNotUsingSA += reg; + nxp_c45_macsec_read(phydev, MACSEC_RXAN0IPUSS + i * 4, ®); + stats->InPktsUnusedSA += reg; + } + + nxp_c45_macsec_read64(phydev, MACSEC_INOD1HS, + &stats->InOctetsDecrypted); + nxp_c45_macsec_read64(phydev, MACSEC_INOV1HS, + &stats->InOctetsValidated); + + nxp_c45_macsec_read32_64(phydev, MACSEC_RXSCIPDS, + &stats->InPktsDelayed); + nxp_c45_macsec_read32_64(phydev, MACSEC_RXSCIPLS, + &stats->InPktsLate); + nxp_c45_macsec_read32_64(phydev, MACSEC_RXSCIPUS, + &stats->InPktsUnchecked); + + return 0; +} + +static int nxp_c45_mdo_get_rx_sa_stats(struct macsec_context *ctx) +{ + struct phy_device *phydev = ctx->phydev; + struct nxp_c45_phy *priv = phydev->priv; + struct macsec_rx_sa_stats *stats; + struct nxp_c45_secy *phy_secy; + u8 an = ctx->sa.assoc_num; + struct nxp_c45_sa *sa; + + phy_secy = nxp_c45_find_secy(&priv->macsec->secy_list, ctx->secy->sci); + if (IS_ERR(phy_secy)) + return PTR_ERR(phy_secy); + + sa = nxp_c45_find_sa(&phy_secy->sa_list, RX_SA, an); + if (IS_ERR(sa)) + return PTR_ERR(sa); + + stats = ctx->stats.rx_sa_stats; + nxp_c45_select_secy(phydev, phy_secy->secy_id); + + nxp_c45_rx_sa_read_stats(phydev, sa, stats); + nxp_c45_macsec_read(phydev, MACSEC_RXAN0INUSS + an * 4, + &stats->InPktsNotUsingSA); + nxp_c45_macsec_read(phydev, MACSEC_RXAN0IPUSS + an * 4, + &stats->InPktsUnusedSA); + + return 0; +} + +struct tja11xx_tlv_header { + struct ethhdr eth; + u8 subtype; + u8 len; + u8 payload[28]; +}; + +static int nxp_c45_mdo_insert_tx_tag(struct phy_device *phydev, + struct sk_buff *skb) +{ + struct tja11xx_tlv_header *tlv; + struct ethhdr *eth; + + eth = eth_hdr(skb); + tlv = skb_push(skb, TJA11XX_TLV_TX_NEEDED_HEADROOM); + memmove(tlv, eth, sizeof(*eth)); + skb_reset_mac_header(skb); + tlv->eth.h_proto = htons(ETH_P_TJA11XX_TLV); + tlv->subtype = 1; + tlv->len = sizeof(tlv->payload); + memset(tlv->payload, 0, sizeof(tlv->payload)); + + return 0; +} + +static const struct macsec_ops nxp_c45_macsec_ops = { + .mdo_dev_open = nxp_c45_mdo_dev_open, + .mdo_dev_stop = nxp_c45_mdo_dev_stop, + .mdo_add_secy = nxp_c45_mdo_add_secy, + .mdo_upd_secy = nxp_c45_mdo_upd_secy, + .mdo_del_secy = nxp_c45_mdo_del_secy, + .mdo_add_rxsc = nxp_c45_mdo_add_rxsc, + .mdo_upd_rxsc = nxp_c45_mdo_upd_rxsc, + .mdo_del_rxsc = nxp_c45_mdo_del_rxsc, + .mdo_add_rxsa = nxp_c45_mdo_add_rxsa, + .mdo_upd_rxsa = nxp_c45_mdo_upd_rxsa, + .mdo_del_rxsa = nxp_c45_mdo_del_rxsa, + .mdo_add_txsa = nxp_c45_mdo_add_txsa, + .mdo_upd_txsa = nxp_c45_mdo_upd_txsa, + .mdo_del_txsa = nxp_c45_mdo_del_txsa, + .mdo_get_dev_stats = nxp_c45_mdo_get_dev_stats, + .mdo_get_tx_sc_stats = nxp_c45_mdo_get_tx_sc_stats, + .mdo_get_tx_sa_stats = nxp_c45_mdo_get_tx_sa_stats, + .mdo_get_rx_sc_stats = nxp_c45_mdo_get_rx_sc_stats, + .mdo_get_rx_sa_stats = nxp_c45_mdo_get_rx_sa_stats, + .mdo_insert_tx_tag = nxp_c45_mdo_insert_tx_tag, + .needed_headroom = TJA11XX_TLV_TX_NEEDED_HEADROOM, + .needed_tailroom = TJA11XX_TLV_NEEDED_TAILROOM, +}; + +int nxp_c45_macsec_config_init(struct phy_device *phydev) +{ + struct nxp_c45_phy *priv = phydev->priv; + int ret; + + if (!priv->macsec) + return 0; + + ret = phy_set_bits_mmd(phydev, MDIO_MMD_VEND1, VEND1_PORT_FUNC_ENABLES, + MACSEC_EN | ADAPTER_EN); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, ADPTR_CNTRL, ADPTR_CNTRL_CONFIG_EN | + ADPTR_CNTRL_ADPTR_EN); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, ADPTR_TX_TAG_CNTRL, + ADPTR_TX_TAG_CNTRL_ENA); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, ADPTR_CNTRL, ADPTR_CNTRL_ADPTR_EN); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, MACSEC_TPNET, PN_WRAP_THRESHOLD); + if (ret) + return ret; + + /* Set MKA filter. */ + ret = nxp_c45_macsec_write(phydev, MACSEC_UPFR0D2, ETH_P_PAE); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, MACSEC_UPFR0M1, MACSEC_OVP); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, MACSEC_UPFR0M2, ETYPE_MASK); + if (ret) + return ret; + + ret = nxp_c45_macsec_write(phydev, MACSEC_UPFR0R, MACSEC_UPFR_EN); + + return ret; +} + +int nxp_c45_macsec_probe(struct phy_device *phydev) +{ + struct nxp_c45_phy *priv = phydev->priv; + struct device *dev = &phydev->mdio.dev; + + priv->macsec = devm_kzalloc(dev, sizeof(*priv->macsec), GFP_KERNEL); + if (!priv->macsec) + return -ENOMEM; + + INIT_LIST_HEAD(&priv->macsec->secy_list); + phydev->macsec_ops = &nxp_c45_macsec_ops; + + return 0; +} + +void nxp_c45_macsec_remove(struct phy_device *phydev) +{ + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *secy_p, *secy_t; + struct nxp_c45_sa *sa_p, *sa_t; + struct list_head *secy_list; + + if (!priv->macsec) + return; + + secy_list = &priv->macsec->secy_list; + nxp_c45_macsec_en(phydev, false); + + list_for_each_entry_safe(secy_p, secy_t, secy_list, list) { + list_for_each_entry_safe(sa_p, sa_t, &secy_p->sa_list, list) + nxp_c45_sa_free(sa_p); + nxp_c45_secy_free(secy_p); + } +} + +void nxp_c45_handle_macsec_interrupt(struct phy_device *phydev, + irqreturn_t *ret) +{ + struct nxp_c45_phy *priv = phydev->priv; + struct nxp_c45_secy *secy; + struct nxp_c45_sa *sa; + u8 encoding_sa; + int secy_id; + u32 reg = 0; + + if (!priv->macsec) + return; + + do { + nxp_c45_macsec_read(phydev, MACSEC_EVR, ®); + if (!reg) + return; + + secy_id = MACSEC_REG_SIZE - ffs(reg); + secy = nxp_c45_find_secy_by_id(&priv->macsec->secy_list, + secy_id); + if (IS_ERR(secy)) { + WARN_ON(1); + goto macsec_ack_irq; + } + + encoding_sa = secy->secy->tx_sc.encoding_sa; + phydev_dbg(phydev, "pn_wrapped: TX SC %d, encoding_sa %u\n", + secy->secy_id, encoding_sa); + + sa = nxp_c45_find_sa(&secy->sa_list, TX_SA, encoding_sa); + if (!IS_ERR(sa)) + macsec_pn_wrapped(secy->secy, sa->sa); + else + WARN_ON(1); + +macsec_ack_irq: + nxp_c45_macsec_write(phydev, MACSEC_EVR, + TX_SC_BIT(secy_id)); + *ret = IRQ_HANDLED; + } while (reg); +} diff --git a/drivers/net/phy/nxp-c45-tja11xx.c b/drivers/net/phy/nxp-c45-tja11xx.c index 780ad353cf55..3cf614b4cd52 100644 --- a/drivers/net/phy/nxp-c45-tja11xx.c +++ b/drivers/net/phy/nxp-c45-tja11xx.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* NXP C45 PHY driver - * Copyright (C) 2021 NXP + * Copyright 2021-2023 NXP * Author: Radu Pirea <[email protected]> */ @@ -14,9 +14,10 @@ #include <linux/processor.h> #include <linux/property.h> #include <linux/ptp_classify.h> -#include <linux/ptp_clock_kernel.h> #include <linux/net_tstamp.h> +#include "nxp-c45-tja11xx.h" + #define PHY_ID_TJA_1103 0x001BB010 #define PHY_ID_TJA_1120 0x001BB031 @@ -75,9 +76,11 @@ #define PORT_CONTROL_EN BIT(14) #define VEND1_PORT_ABILITIES 0x8046 +#define MACSEC_ABILITY BIT(5) #define PTP_ABILITY BIT(3) #define VEND1_PORT_FUNC_IRQ_EN 0x807A +#define MACSEC_IRQS BIT(5) #define PTP_IRQS BIT(3) #define VEND1_PTP_IRQ_ACK 0x9008 @@ -148,7 +151,6 @@ #define TS_SEC_MASK GENMASK(1, 0) -#define VEND1_PORT_FUNC_ENABLES 0x8048 #define PTP_ENABLE BIT(3) #define PHY_TEST_ENABLE BIT(0) @@ -281,25 +283,6 @@ struct nxp_c45_phy_data { irqreturn_t *irq_status); }; -struct nxp_c45_phy { - const struct nxp_c45_phy_data *phy_data; - struct phy_device *phydev; - struct mii_timestamper mii_ts; - struct ptp_clock *ptp_clock; - struct ptp_clock_info caps; - struct sk_buff_head tx_queue; - struct sk_buff_head rx_queue; - /* used to access the PTP registers atomic */ - struct mutex ptp_lock; - int hwts_tx; - int hwts_rx; - u32 tx_delay; - u32 rx_delay; - struct timespec64 extts_ts; - int extts_index; - bool extts; -}; - static const struct nxp_c45_phy_data *nxp_c45_get_data(struct phy_device *phydev) { @@ -1215,12 +1198,25 @@ static int nxp_c45_start_op(struct phy_device *phydev) static int nxp_c45_config_intr(struct phy_device *phydev) { - if (phydev->interrupts == PHY_INTERRUPT_ENABLED) + int ret; + + if (phydev->interrupts == PHY_INTERRUPT_ENABLED) { + ret = phy_set_bits_mmd(phydev, MDIO_MMD_VEND1, + VEND1_PORT_FUNC_IRQ_EN, MACSEC_IRQS); + if (ret) + return ret; + return phy_set_bits_mmd(phydev, MDIO_MMD_VEND1, VEND1_PHY_IRQ_EN, PHY_IRQ_LINK_EVENT); - else - return phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1, - VEND1_PHY_IRQ_EN, PHY_IRQ_LINK_EVENT); + } + + ret = phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1, + VEND1_PORT_FUNC_IRQ_EN, MACSEC_IRQS); + if (ret) + return ret; + + return phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1, + VEND1_PHY_IRQ_EN, PHY_IRQ_LINK_EVENT); } static int tja1103_config_intr(struct phy_device *phydev) @@ -1286,6 +1282,7 @@ static irqreturn_t nxp_c45_handle_interrupt(struct phy_device *phydev) } data->nmi_handler(phydev, &ret); + nxp_c45_handle_macsec_interrupt(phydev, &ret); return ret; } @@ -1611,6 +1608,9 @@ static int nxp_c45_config_init(struct phy_device *phydev) nxp_c45_counters_enable(phydev); nxp_c45_ptp_init(phydev); + ret = nxp_c45_macsec_config_init(phydev); + if (ret) + return ret; return nxp_c45_start_op(phydev); } @@ -1626,7 +1626,9 @@ static int nxp_c45_get_features(struct phy_device *phydev) static int nxp_c45_probe(struct phy_device *phydev) { struct nxp_c45_phy *priv; - int ptp_ability; + bool macsec_ability; + int phy_abilities; + bool ptp_ability; int ret = 0; priv = devm_kzalloc(&phydev->mdio.dev, sizeof(*priv), GFP_KERNEL); @@ -1642,9 +1644,9 @@ static int nxp_c45_probe(struct phy_device *phydev) mutex_init(&priv->ptp_lock); - ptp_ability = phy_read_mmd(phydev, MDIO_MMD_VEND1, - VEND1_PORT_ABILITIES); - ptp_ability = !!(ptp_ability & PTP_ABILITY); + phy_abilities = phy_read_mmd(phydev, MDIO_MMD_VEND1, + VEND1_PORT_ABILITIES); + ptp_ability = !!(phy_abilities & PTP_ABILITY); if (!ptp_ability) { phydev_dbg(phydev, "the phy does not support PTP"); goto no_ptp_support; @@ -1663,6 +1665,20 @@ static int nxp_c45_probe(struct phy_device *phydev) } no_ptp_support: + macsec_ability = !!(phy_abilities & MACSEC_ABILITY); + if (!macsec_ability) { + phydev_info(phydev, "the phy does not support MACsec\n"); + goto no_macsec_support; + } + + if (IS_ENABLED(CONFIG_MACSEC)) { + ret = nxp_c45_macsec_probe(phydev); + phydev_dbg(phydev, "MACsec support enabled."); + } else { + phydev_dbg(phydev, "MACsec support not enabled even if the phy supports it"); + } + +no_macsec_support: return ret; } @@ -1676,6 +1692,7 @@ static void nxp_c45_remove(struct phy_device *phydev) skb_queue_purge(&priv->tx_queue); skb_queue_purge(&priv->rx_queue); + nxp_c45_macsec_remove(phydev); } static void tja1103_counters_enable(struct phy_device *phydev) diff --git a/drivers/net/phy/nxp-c45-tja11xx.h b/drivers/net/phy/nxp-c45-tja11xx.h new file mode 100644 index 000000000000..f364fca68f0b --- /dev/null +++ b/drivers/net/phy/nxp-c45-tja11xx.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* NXP C45 PHY driver header file + * Copyright 2023 NXP + * Author: Radu Pirea <[email protected]> + */ + +#include <linux/ptp_clock_kernel.h> + +#define VEND1_PORT_FUNC_ENABLES 0x8048 + +struct nxp_c45_macsec; + +struct nxp_c45_phy { + const struct nxp_c45_phy_data *phy_data; + struct phy_device *phydev; + struct mii_timestamper mii_ts; + struct ptp_clock *ptp_clock; + struct ptp_clock_info caps; + struct sk_buff_head tx_queue; + struct sk_buff_head rx_queue; + /* used to access the PTP registers atomic */ + struct mutex ptp_lock; + int hwts_tx; + int hwts_rx; + u32 tx_delay; + u32 rx_delay; + struct timespec64 extts_ts; + int extts_index; + bool extts; + struct nxp_c45_macsec *macsec; +}; + +#if IS_ENABLED(CONFIG_MACSEC) +int nxp_c45_macsec_config_init(struct phy_device *phydev); +int nxp_c45_macsec_probe(struct phy_device *phydev); +void nxp_c45_macsec_remove(struct phy_device *phydev); +void nxp_c45_handle_macsec_interrupt(struct phy_device *phydev, + irqreturn_t *ret); +#else +static inline +int nxp_c45_macsec_config_init(struct phy_device *phydev) +{ + return 0; +} + +static inline +int nxp_c45_macsec_probe(struct phy_device *phydev) +{ + return 0; +} + +static inline +void nxp_c45_macsec_remove(struct phy_device *phydev) +{ +} + +static inline +void nxp_c45_handle_macsec_interrupt(struct phy_device *phydev, + irqreturn_t *ret) +{ +} +#endif diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c index 3611ea64875e..1e595762afea 100644 --- a/drivers/net/phy/phy_device.c +++ b/drivers/net/phy/phy_device.c @@ -29,6 +29,7 @@ #include <linux/phy.h> #include <linux/phylib_stubs.h> #include <linux/phy_led_triggers.h> +#include <linux/phy_link_topology.h> #include <linux/pse-pd/pse.h> #include <linux/property.h> #include <linux/rtnetlink.h> @@ -265,6 +266,14 @@ static void phy_mdio_device_remove(struct mdio_device *mdiodev) static struct phy_driver genphy_driver; +static struct phy_link_topology *phy_get_link_topology(struct phy_device *phydev) +{ + if (phydev->attached_dev) + return &phydev->attached_dev->link_topo; + + return NULL; +} + static LIST_HEAD(phy_fixup_list); static DEFINE_MUTEX(phy_fixup_lock); @@ -1355,6 +1364,46 @@ phy_standalone_show(struct device *dev, struct device_attribute *attr, static DEVICE_ATTR_RO(phy_standalone); /** + * phy_sfp_connect_phy - Connect the SFP module's PHY to the upstream PHY + * @upstream: pointer to the upstream phy device + * @phy: pointer to the SFP module's phy device + * + * This helper allows keeping track of PHY devices on the link. It adds the + * SFP module's phy to the phy namespace of the upstream phy + */ +int phy_sfp_connect_phy(void *upstream, struct phy_device *phy) +{ + struct phy_device *phydev = upstream; + struct phy_link_topology *topo = phy_get_link_topology(phydev); + + if (topo) + return phy_link_topo_add_phy(topo, phy, PHY_UPSTREAM_PHY, phydev); + + return 0; +} +EXPORT_SYMBOL(phy_sfp_connect_phy); + +/** + * phy_sfp_disconnect_phy - Disconnect the SFP module's PHY from the upstream PHY + * @upstream: pointer to the upstream phy device + * @phy: pointer to the SFP module's phy device + * + * This helper allows keeping track of PHY devices on the link. It removes the + * SFP module's phy to the phy namespace of the upstream phy. As the module phy + * will be destroyed, re-inserting the same module will add a new phy with a + * new index. + */ +void phy_sfp_disconnect_phy(void *upstream, struct phy_device *phy) +{ + struct phy_device *phydev = upstream; + struct phy_link_topology *topo = phy_get_link_topology(phydev); + + if (topo) + phy_link_topo_del_phy(topo, phy); +} +EXPORT_SYMBOL(phy_sfp_disconnect_phy); + +/** * phy_sfp_attach - attach the SFP bus to the PHY upstream network device * @upstream: pointer to the phy device * @bus: sfp bus representing cage being attached @@ -1491,6 +1540,11 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev, if (phydev->sfp_bus_attached) dev->sfp_bus = phydev->sfp_bus; + + err = phy_link_topo_add_phy(&dev->link_topo, phydev, + PHY_UPSTREAM_MAC, dev); + if (err) + goto error; } /* Some Ethernet drivers try to connect to a PHY device before @@ -1820,6 +1874,7 @@ void phy_detach(struct phy_device *phydev) if (dev) { phydev->attached_dev->phydev = NULL; phydev->attached_dev = NULL; + phy_link_topo_del_phy(&dev->link_topo, phydev); } phydev->phylink = NULL; diff --git a/drivers/net/phy/phy_link_topology.c b/drivers/net/phy/phy_link_topology.c new file mode 100644 index 000000000000..34e7e08fbfc3 --- /dev/null +++ b/drivers/net/phy/phy_link_topology.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Infrastructure to handle all PHY devices connected to a given netdev, + * either directly or indirectly attached. + * + * Copyright (c) 2023 Maxime Chevallier<[email protected]> + */ + +#include <linux/phy_link_topology.h> +#include <linux/netdevice.h> +#include <linux/phy.h> +#include <linux/rtnetlink.h> +#include <linux/xarray.h> + +int phy_link_topo_add_phy(struct phy_link_topology *topo, + struct phy_device *phy, + enum phy_upstream upt, void *upstream) +{ + struct phy_device_node *pdn; + int ret; + + pdn = kzalloc(sizeof(*pdn), GFP_KERNEL); + if (!pdn) + return -ENOMEM; + + pdn->phy = phy; + switch (upt) { + case PHY_UPSTREAM_MAC: + pdn->upstream.netdev = (struct net_device *)upstream; + if (phy_on_sfp(phy)) + pdn->parent_sfp_bus = pdn->upstream.netdev->sfp_bus; + break; + case PHY_UPSTREAM_PHY: + pdn->upstream.phydev = (struct phy_device *)upstream; + if (phy_on_sfp(phy)) + pdn->parent_sfp_bus = pdn->upstream.phydev->sfp_bus; + break; + default: + ret = -EINVAL; + goto err; + } + pdn->upstream_type = upt; + + ret = xa_alloc_cyclic(&topo->phys, &phy->phyindex, pdn, xa_limit_32b, + &topo->next_phy_index, GFP_KERNEL); + if (ret) + goto err; + + return 0; + +err: + kfree(pdn); + return ret; +} +EXPORT_SYMBOL_GPL(phy_link_topo_add_phy); + +void phy_link_topo_del_phy(struct phy_link_topology *topo, + struct phy_device *phy) +{ + struct phy_device_node *pdn = xa_erase(&topo->phys, phy->phyindex); + + phy->phyindex = 0; + + kfree(pdn); +} +EXPORT_SYMBOL_GPL(phy_link_topo_del_phy); diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c index 298dfd6982a5..3d25a4a6212b 100644 --- a/drivers/net/phy/phylink.c +++ b/drivers/net/phy/phylink.c @@ -3319,7 +3319,8 @@ static int phylink_sfp_connect_phy(void *upstream, struct phy_device *phy) return ret; } -static void phylink_sfp_disconnect_phy(void *upstream) +static void phylink_sfp_disconnect_phy(void *upstream, + struct phy_device *phydev) { phylink_disconnect_phy(upstream); } diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c index 6fa679b36290..fb1c102714b5 100644 --- a/drivers/net/phy/sfp-bus.c +++ b/drivers/net/phy/sfp-bus.c @@ -486,7 +486,7 @@ static void sfp_unregister_bus(struct sfp_bus *bus) bus->socket_ops->stop(bus->sfp); bus->socket_ops->detach(bus->sfp); if (bus->phydev && ops && ops->disconnect_phy) - ops->disconnect_phy(bus->upstream); + ops->disconnect_phy(bus->upstream, bus->phydev); } bus->registered = false; } @@ -742,7 +742,7 @@ void sfp_remove_phy(struct sfp_bus *bus) const struct sfp_upstream_ops *ops = sfp_get_upstream_ops(bus); if (ops && ops->disconnect_phy) - ops->disconnect_phy(bus->upstream); + ops->disconnect_phy(bus->upstream, bus->phydev); bus->phydev = NULL; } EXPORT_SYMBOL_GPL(sfp_remove_phy); @@ -859,3 +859,14 @@ void sfp_unregister_socket(struct sfp_bus *bus) sfp_bus_put(bus); } EXPORT_SYMBOL_GPL(sfp_unregister_socket); + +const char *sfp_get_name(struct sfp_bus *bus) +{ + ASSERT_RTNL(); + + if (bus->sfp_dev) + return dev_name(bus->sfp_dev); + + return NULL; +} +EXPORT_SYMBOL_GPL(sfp_get_name); diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c index 3780a96d2caa..f75c9eb3958e 100644 --- a/drivers/net/phy/sfp.c +++ b/drivers/net/phy/sfp.c @@ -274,6 +274,7 @@ struct sfp { struct sfp_eeprom_id id; unsigned int module_power_mW; unsigned int module_t_start_up; + unsigned int module_t_wait; unsigned int phy_t_retry; unsigned int rate_kbd; @@ -388,6 +389,12 @@ static void sfp_fixup_fs_10gt(struct sfp *sfp) { sfp_fixup_10gbaset_30m(sfp); sfp_fixup_rollball(sfp); + + /* The RollBall fixup is not enough for FS modules, the AQR chip inside + * them does not return 0xffff for PHY ID registers in all MMDs for the + * while initializing. They need a 4 second wait before accessing PHY. + */ + sfp->module_t_wait = msecs_to_jiffies(4000); } static void sfp_fixup_halny_gsfp(struct sfp *sfp) @@ -2329,6 +2336,7 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report) mask |= SFP_F_RS1; sfp->module_t_start_up = T_START_UP; + sfp->module_t_wait = T_WAIT; sfp->phy_t_retry = T_PHY_RETRY; sfp->state_ignore_mask = 0; @@ -2566,9 +2574,10 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) /* We need to check the TX_FAULT state, which is not defined * while TX_DISABLE is asserted. The earliest we want to do - * anything (such as probe for a PHY) is 50ms. + * anything (such as probe for a PHY) is 50ms (or more on + * specific modules). */ - sfp_sm_next(sfp, SFP_S_WAIT, T_WAIT); + sfp_sm_next(sfp, SFP_S_WAIT, sfp->module_t_wait); break; case SFP_S_WAIT: @@ -2582,8 +2591,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) * deasserting. */ timeout = sfp->module_t_start_up; - if (timeout > T_WAIT) - timeout -= T_WAIT; + if (timeout > sfp->module_t_wait) + timeout -= sfp->module_t_wait; else timeout = 1; diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c index fa0f598ed6bf..7d28ae5453cf 100644 --- a/drivers/net/wireless/ath/ath10k/htt_rx.c +++ b/drivers/net/wireless/ath/ath10k/htt_rx.c @@ -1295,7 +1295,7 @@ static void ath10k_htt_rx_h_ppdu(struct ath10k *ar, status->encoding = RX_ENC_LEGACY; status->bw = RATE_INFO_BW_20; - status->flag &= ~RX_FLAG_MACTIME_END; + status->flag &= ~RX_FLAG_MACTIME; status->flag |= RX_FLAG_NO_SIGNAL_VAL; status->flag &= ~(RX_FLAG_AMPDU_IS_LAST); diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h b/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h index 7b18e098b125..798731ecbefd 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h @@ -60,6 +60,12 @@ enum iwl_debug_cmds { */ FW_DUMP_COMPLETE_CMD = 0xB, /** + * @FW_CLEAR_BUFFER: + * clears the firmware's internal buffer + * no payload + */ + FW_CLEAR_BUFFER = 0xD, + /** * @MFU_ASSERT_DUMP_NTF: * &struct iwl_mfu_assert_dump_notif */ diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h index dfe0bebabc81..7ec959244ffc 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h @@ -269,6 +269,9 @@ struct iwl_nvm_access_complete_cmd { __le32 reserved; } __packed; /* NVM_ACCESS_COMPLETE_CMD_API_S_VER_1 */ +#define IWL_MCC_US 0x5553 +#define IWL_MCC_CANADA 0x4341 + /** * struct iwl_mcc_update_cmd - Request the device to update geographic * regulatory profile according to the given MCC (Mobile Country Code). diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c index f6e399d1e95c..e27774e7ed74 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c @@ -3401,6 +3401,15 @@ void iwl_fw_dbg_clear_monitor_buf(struct iwl_fw_runtime *fwrt) struct iwl_fw_dbg_params params = {0}; iwl_fw_dbg_stop_sync(fwrt); + + if (fw_has_api(&fwrt->fw->ucode_capa, + IWL_UCODE_TLV_API_INT_DBG_BUF_CLEAR)) { + struct iwl_host_cmd hcmd = { + .id = WIDE_ID(DEBUG_GROUP, FW_CLEAR_BUFFER), + }; + iwl_trans_send_cmd(fwrt->trans, &hcmd); + } + iwl_dbg_tlv_init_cfg(fwrt); iwl_fw_dbg_stop_restart_recording(fwrt, ¶ms, false); } diff --git a/drivers/net/wireless/intel/iwlwifi/fw/file.h b/drivers/net/wireless/intel/iwlwifi/fw/file.h index c38e5194c55f..bfc39bd5bbc6 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/file.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/file.h @@ -245,6 +245,8 @@ typedef unsigned int __bitwise iwl_ucode_tlv_api_t; * SCAN_CONFIG_DB_CMD_API_S. * @IWL_UCODE_TLV_API_NO_HOST_DISABLE_TX: Firmware offloaded the station disable tx * logic. + * @IWL_UCODE_TLV_API_INT_DBG_BUF_CLEAR: Firmware supports clearing the debug + * internal buffer * * @NUM_IWL_UCODE_TLV_API: number of bits used */ @@ -282,7 +284,9 @@ enum iwl_ucode_tlv_api { IWL_UCODE_TLV_API_ADWELL_HB_DEF_N_AP = (__force iwl_ucode_tlv_api_t)57, IWL_UCODE_TLV_API_SCAN_EXT_CHAN_VER = (__force iwl_ucode_tlv_api_t)58, IWL_UCODE_TLV_API_BAND_IN_RX_DATA = (__force iwl_ucode_tlv_api_t)59, + /* API Set 2 */ IWL_UCODE_TLV_API_NO_HOST_DISABLE_TX = (__force iwl_ucode_tlv_api_t)66, + IWL_UCODE_TLV_API_INT_DBG_BUF_CLEAR = (__force iwl_ucode_tlv_api_t)67, NUM_IWL_UCODE_TLV_API /* diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h index 02ded22295c1..ae6f1cd4d660 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h @@ -377,7 +377,6 @@ struct iwl_cfg { u16 nvm_calib_ver; u32 rx_with_siso_diversity:1, tx_with_siso_diversity:1, - bt_shared_single_ant:1, internal_wimax_coex:1, host_interrupt_operation_mode:1, high_temp:1, diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h index a4df67ff21ba..4511d7fb2279 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h @@ -354,6 +354,8 @@ enum { #define CSR_HW_RF_ID_TYPE_GF (0x0010D000) #define CSR_HW_RF_ID_TYPE_GF4 (0x0010E000) #define CSR_HW_RF_ID_TYPE_MS (0x00111000) +#define CSR_HW_RF_ID_TYPE_FM (0x00112000) +#define CSR_HW_RF_ID_TYPE_WP (0x00113000) /* HW_RF CHIP STEP */ #define CSR_HW_RF_STEP(_val) (((_val) >> 8) & 0xF) diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c index 480f8edbfd35..402896988686 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c @@ -1609,10 +1609,17 @@ static u32 iwl_nvm_get_regdom_bw_flags(const u16 *nvm_chan, /* Set the GO concurrent flag only in case that NO_IR is set. * Otherwise it is meaningless */ - if ((nvm_flags & NVM_CHANNEL_GO_CONCURRENT) && - (flags & NL80211_RRF_NO_IR)) - flags |= NL80211_RRF_GO_CONCURRENT; - + if ((nvm_flags & NVM_CHANNEL_GO_CONCURRENT)) { + if (flags & NL80211_RRF_NO_IR) + flags |= NL80211_RRF_GO_CONCURRENT; + if (flags & NL80211_RRF_DFS) { + flags |= NL80211_RRF_DFS_CONCURRENT; + /* Our device doesn't set active bit for DFS channels + * however, once marked as DFS no-ir is not needed. + */ + flags &= ~NL80211_RRF_NO_IR; + } + } /* * reg_capa is per regulatory domain so apply it for every channel */ diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h index ef7dc0a7b56c..5789a8735976 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h @@ -1316,7 +1316,7 @@ iwl_trans_get_rxq_dma_data(struct iwl_trans *trans, int queue, struct iwl_trans_rxq_dma_data *data) { if (WARN_ON_ONCE(!trans->ops->rxq_dma_data)) - return -ENOTSUPP; + return -EOPNOTSUPP; return trans->ops->rxq_dma_data(trans, queue, data); } @@ -1338,7 +1338,7 @@ iwl_trans_txq_alloc(struct iwl_trans *trans, might_sleep(); if (WARN_ON_ONCE(!trans->ops->txq_alloc)) - return -ENOTSUPP; + return -EOPNOTSUPP; if (WARN_ON_ONCE(trans->state != IWL_TRANS_FW_ALIVE)) { IWL_ERR(trans, "%s bad state = %d\n", __func__, trans->state); @@ -1404,7 +1404,7 @@ static inline int iwl_trans_wait_tx_queues_empty(struct iwl_trans *trans, u32 txqs) { if (WARN_ON_ONCE(!trans->ops->wait_tx_queues_empty)) - return -ENOTSUPP; + return -EOPNOTSUPP; /* No need to wait if the firmware is not alive */ if (trans->state != IWL_TRANS_FW_ALIVE) { @@ -1418,7 +1418,7 @@ static inline int iwl_trans_wait_tx_queues_empty(struct iwl_trans *trans, static inline int iwl_trans_wait_txq_empty(struct iwl_trans *trans, int queue) { if (WARN_ON_ONCE(!trans->ops->wait_txq_empty)) - return -ENOTSUPP; + return -EOPNOTSUPP; if (WARN_ON_ONCE(trans->state != IWL_TRANS_FW_ALIVE)) { IWL_ERR(trans, "%s bad state = %d\n", __func__, trans->state); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/coex.c b/drivers/net/wireless/intel/iwlwifi/mvm/coex.c index 5a5b1128e75c..9fe1761691ec 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/coex.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/coex.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright (C) 2013-2014, 2018-2020, 2022 Intel Corporation + * Copyright (C) 2013-2014, 2018-2020, 2022-2023 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH */ #include <linux/ieee80211.h> @@ -116,11 +116,6 @@ iwl_get_coex_type(struct iwl_mvm *mvm, const struct ieee80211_vif *vif) ret = BT_COEX_TX_DIS_LUT; - if (mvm->cfg->bt_shared_single_ant) { - rcu_read_unlock(); - return ret; - } - phy_ctx_id = *((u16 *)chanctx_conf->drv_priv); primary_ch_phy_id = le32_to_cpu(mvm->last_bt_ci_cmd.primary_ch_phy_id); secondary_ch_phy_id = @@ -383,13 +378,12 @@ static void iwl_mvm_bt_notif_per_link(struct iwl_mvm *mvm, /* * don't reduce the Tx power if one of these is true: * we are in LOOSE - * single share antenna product * BT is inactive * we are not associated */ if (iwl_get_coex_type(mvm, vif) == BT_COEX_LOOSE_LUT || - mvm->cfg->bt_shared_single_ant || !vif->cfg.assoc || - le32_to_cpu(mvm->last_bt_notif.bt_activity_grading) == BT_OFF) { + le32_to_cpu(mvm->last_bt_notif.bt_activity_grading) == BT_OFF || + !vif->cfg.assoc) { iwl_mvm_bt_coex_reduced_txp(mvm, link_info->ap_sta_id, false); /* FIXME: should this be per link? */ iwl_mvm_bt_coex_enable_rssi_event(mvm, vif, false, 0); @@ -570,7 +564,7 @@ void iwl_mvm_bt_rssi_event(struct iwl_mvm *mvm, struct ieee80211_vif *vif, * Check if rssi is good enough for reduced Tx power, but not in loose * scheme. */ - if (rssi_event == RSSI_EVENT_LOW || mvm->cfg->bt_shared_single_ant || + if (rssi_event == RSSI_EVENT_LOW || iwl_get_coex_type(mvm, vif) == BT_COEX_LOOSE_LUT) ret = iwl_mvm_bt_coex_reduced_txp(mvm, mvmvif->deflink.ap_sta_id, @@ -639,10 +633,6 @@ bool iwl_mvm_bt_coex_is_mimo_allowed(struct iwl_mvm *mvm, bool iwl_mvm_bt_coex_is_ant_avail(struct iwl_mvm *mvm, u8 ant) { - /* there is no other antenna, shared antenna is always available */ - if (mvm->cfg->bt_shared_single_ant) - return true; - if (ant & mvm->cfg->non_shared_ant) return true; @@ -652,10 +642,6 @@ bool iwl_mvm_bt_coex_is_ant_avail(struct iwl_mvm *mvm, u8 ant) bool iwl_mvm_bt_coex_is_shared_ant_avail(struct iwl_mvm *mvm) { - /* there is no other antenna, shared antenna is always available */ - if (mvm->cfg->bt_shared_single_ant) - return true; - return le32_to_cpu(mvm->last_bt_notif.bt_activity_grading) < BT_HIGH_TRAFFIC; } diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c index 92c45571bd69..4582afb149d7 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c @@ -1130,14 +1130,7 @@ iwl_mvm_wowlan_config(struct iwl_mvm *mvm, return ret; } - /* - * This needs to be unlocked due to lock ordering - * constraints. Since we're in the suspend path - * that isn't really a problem though. - */ - mutex_unlock(&mvm->mutex); ret = iwl_mvm_wowlan_config_key_params(mvm, vif); - mutex_lock(&mvm->mutex); if (ret) return ret; @@ -2497,7 +2490,7 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm, struct iwl_wowlan_status_data *status) { int i; - bool keep; + bool keep = false; struct iwl_mvm_sta *mvm_ap_sta; if (!status) @@ -2525,18 +2518,12 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm, mvm_ap_sta->tid_data[i].seq_number >> 4); } - /* now we have all the data we need, unlock to avoid mac80211 issues */ - mutex_unlock(&mvm->mutex); - iwl_mvm_report_wakeup_reasons(mvm, vif, status); keep = iwl_mvm_setup_connection_keep(mvm, vif, status); - - return keep; - out_unlock: mutex_unlock(&mvm->mutex); - return false; + return keep; } #define ND_QUERY_BUF_LEN (sizeof(struct iwl_scan_offload_profile_match) * \ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c index e016fce7ab24..edc8204f7c0e 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c @@ -1521,7 +1521,7 @@ static ssize_t iwl_dbgfs_inject_packet_write(struct iwl_mvm *mvm, /* supporting only MQ RX */ if (!mvm->trans->trans_cfg->mq_rx_supported) - return -ENOTSUPP; + return -EOPNOTSUPP; rxb._page = alloc_pages(GFP_ATOMIC, 0); if (!rxb._page) @@ -1829,7 +1829,7 @@ static ssize_t _iwl_dbgfs_link_sta_##name##_write(struct file *file, \ char buf[buflen] = {}; \ size_t buf_size = min(count, sizeof(buf) - 1); \ \ - if (copy_from_user(buf, user_buf, sizeof(buf))) \ + if (copy_from_user(buf, user_buf, buf_size)) \ return -EFAULT; \ \ return _iwl_dbgfs_link_sta_wrap_write(iwl_dbgfs_##name##_write, \ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c index 10b9219b3bfd..8f10590f9cdd 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c @@ -39,7 +39,7 @@ static int iwl_mvm_ftm_responder_set_bw_v1(struct cfg80211_chan_def *chandef, *ctrl_ch_position = iwl_mvm_get_ctrl_pos(chandef); break; default: - return -ENOTSUPP; + return -EOPNOTSUPP; } return 0; @@ -77,7 +77,7 @@ static int iwl_mvm_ftm_responder_set_bw_v2(struct cfg80211_chan_def *chandef, } fallthrough; default: - return -ENOTSUPP; + return -EOPNOTSUPP; } return 0; @@ -291,7 +291,7 @@ iwl_mvm_ftm_responder_dyn_cfg_cmd(struct iwl_mvm *mvm, default: IWL_ERR(mvm, "Unsupported DYN_CONFIG_CMD version %u\n", cmd_ver); - ret = -ENOTSUPP; + ret = -EOPNOTSUPP; } return ret; @@ -333,7 +333,7 @@ int iwl_mvm_ftm_respoder_add_pasn_sta(struct iwl_mvm *mvm, if (cmd_ver < 3) { IWL_ERR(mvm, "Adding PASN station not supported by FW\n"); - return -ENOTSUPP; + return -EOPNOTSUPP; } if ((!hltk || !hltk_len) && (!tk || !tk_len)) { diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c index 403bd17b8b7a..1252084662c6 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c @@ -27,9 +27,6 @@ #define MVM_UCODE_ALIVE_TIMEOUT (2 * HZ) #define MVM_UCODE_CALIB_TIMEOUT (2 * HZ) -#define IWL_TAS_US_MCC 0x5553 -#define IWL_TAS_CANADA_MCC 0x4341 - #define IWL_UATS_VLP_AP_SUPPORTED BIT(29) #define IWL_UATS_AFC_AP_SUPPORTED BIT(30) @@ -1234,10 +1231,10 @@ static void iwl_mvm_tas_init(struct iwl_mvm *mvm) dmi_get_system_info(DMI_SYS_VENDOR)); if ((!iwl_mvm_add_to_tas_block_list(cmd.v4.block_list_array, &cmd.v4.block_list_size, - IWL_TAS_US_MCC)) || + IWL_MCC_US)) || (!iwl_mvm_add_to_tas_block_list(cmd.v4.block_list_array, &cmd.v4.block_list_size, - IWL_TAS_CANADA_MCC))) { + IWL_MCC_CANADA))) { IWL_DEBUG_RADIO(mvm, "Unable to add US/Canada to TAS block list, disabling TAS\n"); return; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c index a64600f0ed9f..7f13dff04b26 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c @@ -152,6 +152,16 @@ struct ieee80211_regdomain *iwl_mvm_get_regdomain(struct wiphy *wiphy, mvm->lar_regdom_set = true; mvm->mcc_src = src_id; + /* Some kind of regulatory mess means we need to currently disallow + * puncturing in the US and Canada. Do that here, at least until we + * figure out the new chanctx APIs for puncturing. + */ + if (resp->mcc == cpu_to_le16(IWL_MCC_US) || + resp->mcc == cpu_to_le16(IWL_MCC_CANADA)) + ieee80211_hw_set(mvm->hw, DISALLOW_PUNCTURING); + else + __clear_bit(IEEE80211_HW_DISALLOW_PUNCTURING, mvm->hw->flags); + iwl_mei_set_country_code(__le16_to_cpu(resp->mcc)); out: @@ -288,7 +298,7 @@ int iwl_mvm_op_set_antenna(struct ieee80211_hw *hw, u32 tx_ant, u32 rx_ant) /* This has been tested on those devices only */ if (mvm->trans->trans_cfg->device_family != IWL_DEVICE_FAMILY_9000 && mvm->trans->trans_cfg->device_family != IWL_DEVICE_FAMILY_22000) - return -ENOTSUPP; + return -EOPNOTSUPP; if (!mvm->nvm_data) return -EBUSY; @@ -517,6 +527,10 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm) hw->wiphy->regulatory_flags |= REGULATORY_CUSTOM_REG | REGULATORY_DISABLE_BEACON_HINTS; + if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) + wiphy_ext_feature_set(hw->wiphy, + NL80211_EXT_FEATURE_DFS_CONCURRENT); + hw->wiphy->flags |= WIPHY_FLAG_AP_UAPSD; hw->wiphy->flags |= WIPHY_FLAG_HAS_CHANNEL_SWITCH; hw->wiphy->flags |= WIPHY_FLAG_SPLIT_SCAN_6GHZ; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c index ff6cb064051b..61170173f917 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c @@ -271,17 +271,17 @@ __iwl_mvm_mld_assign_vif_chanctx(struct iwl_mvm *mvm, } } + mvmvif->link[link_id]->phy_ctxt = phy_ctxt; + if (iwl_mvm_is_esr_supported(mvm->fwrt.trans) && n_active > 1) { mvmvif->link[link_id]->listen_lmac = true; ret = iwl_mvm_esr_mode_active(mvm, vif); if (ret) { IWL_ERR(mvm, "failed to activate ESR mode (%d)\n", ret); - return ret; + goto out; } } - mvmvif->link[link_id]->phy_ctxt = phy_ctxt; - if (switching_chanctx) { /* reactivate if we turned this off during channel switch */ if (vif->type == NL80211_IFTYPE_AP) @@ -716,7 +716,7 @@ void iwl_mvm_mld_select_links(struct iwl_mvm *mvm, struct ieee80211_vif *vif, } } - if (WARN_ON(!new_active_links)) + if (!new_active_links) return; if (vif->active_links != new_active_links) diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c index 4e1fccff3987..334d1f59f6e4 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c @@ -99,17 +99,6 @@ static void iwl_mvm_phy_ctxt_set_rxchain(struct iwl_mvm *mvm, active_cnt = 2; } - /* - * If the firmware requested it, then we know that it supports - * getting zero for the values to indicate "use one, but pick - * which one yourself", which means it can dynamically pick one - * that e.g. has better RSSI. - */ - if (mvm->fw_static_smps_request && active_cnt == 1 && idle_cnt == 1) { - idle_cnt = 0; - active_cnt = 0; - } - *rxchain_info = cpu_to_le32(iwl_mvm_get_valid_rx_ant(mvm) << PHY_RX_CHAIN_VALID_POS); *rxchain_info |= cpu_to_le32(idle_cnt << PHY_RX_CHAIN_CNT_POS); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c index efe3e111ea0a..2a3ca9785974 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c @@ -2550,7 +2550,7 @@ int iwl_mvm_add_mcast_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif) if (WARN_ON(vif->type != NL80211_IFTYPE_AP && vif->type != NL80211_IFTYPE_ADHOC)) - return -ENOTSUPP; + return -EOPNOTSUPP; /* * In IBSS, ieee80211_check_queues() sets the cab_queue to be @@ -3234,7 +3234,7 @@ int iwl_mvm_sta_tx_agg_oper(struct iwl_mvm *mvm, struct ieee80211_vif *vif, * should be updated as well. */ if (buf_size < IWL_FRAME_LIMIT) - return -ENOTSUPP; + return -EOPNOTSUPP; ret = iwl_mvm_sta_tx_agg(mvm, sta, tid, queue, true); if (ret) diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c index ae5cd13cd6dd..db986bfc4dc3 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c @@ -2256,7 +2256,7 @@ int iwl_mvm_flush_sta_tids(struct iwl_mvm *mvm, u32 sta_id, u16 tids) WARN_ON(!iwl_mvm_has_new_tx_api(mvm)); if (iwl_fw_lookup_notif_ver(mvm->fw, LONG_GROUP, TXPATH_FLUSH, 0) > 0) - cmd.flags |= CMD_WANT_SKB; + cmd.flags |= CMD_WANT_SKB | CMD_SEND_IN_RFKILL; IWL_DEBUG_TX_QUEUES(mvm, "flush for sta id %d tid mask 0x%x\n", sta_id, tids); diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c index c9e5bda8f0b7..a4a4772330cf 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c @@ -290,6 +290,16 @@ static void iwl_pcie_get_rf_name(struct iwl_trans *trans) case CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_MS): pos = scnprintf(buf, buflen, "MS"); break; + case CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_FM): + pos = scnprintf(buf, buflen, "FM"); + break; + case CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_WP): + if (SILICON_Z_STEP == + CSR_HW_RFID_STEP(trans->hw_rf_id)) + pos = scnprintf(buf, buflen, "WHTC"); + else + pos = scnprintf(buf, buflen, "WH"); + break; default: return; } diff --git a/drivers/net/wireless/marvell/mwifiex/join.c b/drivers/net/wireless/marvell/mwifiex/join.c index a6e254a1185c..9d98a1908dd6 100644 --- a/drivers/net/wireless/marvell/mwifiex/join.c +++ b/drivers/net/wireless/marvell/mwifiex/join.c @@ -1427,8 +1427,8 @@ int mwifiex_adhoc_join(struct mwifiex_private *priv, /* Check if the requested SSID is already joined */ if (priv->curr_bss_params.bss_descriptor.ssid.ssid_len && - !mwifiex_ssid_cmp(&bss_desc->ssid, - &priv->curr_bss_params.bss_descriptor.ssid) && + cfg80211_ssid_eq(&bss_desc->ssid, + &priv->curr_bss_params.bss_descriptor.ssid) && (priv->curr_bss_params.bss_descriptor.bss_mode == NL80211_IFTYPE_ADHOC)) { mwifiex_dbg(priv->adapter, INFO, diff --git a/drivers/net/wireless/marvell/mwifiex/main.h b/drivers/net/wireless/marvell/mwifiex/main.h index d263eae6078c..318b42b1896f 100644 --- a/drivers/net/wireless/marvell/mwifiex/main.h +++ b/drivers/net/wireless/marvell/mwifiex/main.h @@ -1152,7 +1152,6 @@ void mwifiex_queue_scan_cmd(struct mwifiex_private *priv, struct cmd_ctrl_node *cmd_node); int mwifiex_ret_802_11_scan(struct mwifiex_private *priv, struct host_cmd_ds_command *resp); -s32 mwifiex_ssid_cmp(struct cfg80211_ssid *ssid1, struct cfg80211_ssid *ssid2); int mwifiex_associate(struct mwifiex_private *priv, struct mwifiex_bssdescriptor *bss_desc); int mwifiex_cmd_802_11_associate(struct mwifiex_private *priv, diff --git a/drivers/net/wireless/marvell/mwifiex/scan.c b/drivers/net/wireless/marvell/mwifiex/scan.c index 72904c275461..a2ddac363b10 100644 --- a/drivers/net/wireless/marvell/mwifiex/scan.c +++ b/drivers/net/wireless/marvell/mwifiex/scan.c @@ -180,17 +180,6 @@ mwifiex_is_wpa_oui_present(struct mwifiex_bssdescriptor *bss_desc, u32 cipher) } /* - * This function compares two SSIDs and checks if they match. - */ -s32 -mwifiex_ssid_cmp(struct cfg80211_ssid *ssid1, struct cfg80211_ssid *ssid2) -{ - if (!ssid1 || !ssid2 || (ssid1->ssid_len != ssid2->ssid_len)) - return -1; - return memcmp(ssid1->ssid, ssid2->ssid, ssid1->ssid_len); -} - -/* * This function checks if wapi is enabled in driver and scanned network is * compatible with it. */ diff --git a/drivers/net/wireless/marvell/mwifiex/sdio.c b/drivers/net/wireless/marvell/mwifiex/sdio.c index ef3e68d1059c..75f53c2f1e1f 100644 --- a/drivers/net/wireless/marvell/mwifiex/sdio.c +++ b/drivers/net/wireless/marvell/mwifiex/sdio.c @@ -779,7 +779,7 @@ static int mwifiex_check_fw_status(struct mwifiex_adapter *adapter, { struct sdio_mmc_card *card = adapter->card; int ret = 0; - u16 firmware_stat; + u16 firmware_stat = 0; u32 tries; for (tries = 0; tries < poll_num; tries++) { diff --git a/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c b/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c index a2ad2b53f016..32a27fad7b79 100644 --- a/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c +++ b/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c @@ -345,8 +345,8 @@ int mwifiex_bss_start(struct mwifiex_private *priv, struct cfg80211_bss *bss, /* Adhoc mode */ /* If the requested SSID matches current SSID, return */ if (bss_desc && bss_desc->ssid.ssid_len && - (!mwifiex_ssid_cmp(&priv->curr_bss_params.bss_descriptor. - ssid, &bss_desc->ssid))) { + cfg80211_ssid_eq(&priv->curr_bss_params.bss_descriptor.ssid, + &bss_desc->ssid)) { ret = 0; goto done; } diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c index 43ee7592bc6e..180907319e8c 100644 --- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c +++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c @@ -7961,6 +7961,18 @@ static const struct usb_device_id dev_table[] = { .driver_info = (unsigned long)&rtl8192eu_fops}, {USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x818c, 0xff, 0xff, 0xff), .driver_info = (unsigned long)&rtl8192eu_fops}, +/* D-Link DWA-131 rev C1 */ +{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3312, 0xff, 0xff, 0xff), + .driver_info = (unsigned long)&rtl8192eu_fops}, +/* TP-Link TL-WN8200ND V2 */ +{USB_DEVICE_AND_INTERFACE_INFO(0x2357, 0x0126, 0xff, 0xff, 0xff), + .driver_info = (unsigned long)&rtl8192eu_fops}, +/* Mercusys MW300UM */ +{USB_DEVICE_AND_INTERFACE_INFO(0x2c4e, 0x0100, 0xff, 0xff, 0xff), + .driver_info = (unsigned long)&rtl8192eu_fops}, +/* Mercusys MW300UH */ +{USB_DEVICE_AND_INTERFACE_INFO(0x2c4e, 0x0104, 0xff, 0xff, 0xff), + .driver_info = (unsigned long)&rtl8192eu_fops}, #endif { } }; diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/phy.c index 12d0b3a87af7..0fab3a0c7d49 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/phy.c @@ -16,12 +16,6 @@ static u32 _rtl88e_phy_rf_serial_read(struct ieee80211_hw *hw, static void _rtl88e_phy_rf_serial_write(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset, u32 data); -static u32 _rtl88e_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} static bool _rtl88e_phy_bb8188e_config_parafile(struct ieee80211_hw *hw); static bool _rtl88e_phy_config_mac_with_headerfile(struct ieee80211_hw *hw); static bool phy_config_bb_with_headerfile(struct ieee80211_hw *hw, @@ -51,7 +45,7 @@ u32 rtl88e_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), bitmask(%#x)\n", regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl88e_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -74,7 +68,7 @@ void rtl88e_phy_set_bb_reg(struct ieee80211_hw *hw, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl88e_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } @@ -99,7 +93,7 @@ u32 rtl88e_phy_query_rf_reg(struct ieee80211_hw *hw, original_value = _rtl88e_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl88e_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -127,7 +121,7 @@ void rtl88e_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = _rtl88e_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl88e_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.c index d448efe2c229..3730613a3962 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.c @@ -17,7 +17,7 @@ u32 rtl92c_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), bitmask(%#x)\n", regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -40,7 +40,7 @@ void rtl92c_phy_set_bb_reg(struct ieee80211_hw *hw, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } @@ -143,14 +143,6 @@ void _rtl92c_phy_rf_serial_write(struct ieee80211_hw *hw, } EXPORT_SYMBOL(_rtl92c_phy_rf_serial_write); -u32 _rtl92c_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} -EXPORT_SYMBOL(_rtl92c_phy_calculate_bit_shift); - static void _rtl92c_phy_bb_config_1t(struct ieee80211_hw *hw) { rtl_set_bbreg(hw, RFPGA0_TXINFO, 0x3, 0x2); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.h index 75afa6253ad0..e64d377dfe9e 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192c/phy_common.h @@ -196,7 +196,6 @@ bool rtl92c_phy_set_rf_power_state(struct ieee80211_hw *hw, void rtl92ce_phy_set_rf_on(struct ieee80211_hw *hw); void rtl92c_phy_set_io(struct ieee80211_hw *hw); void rtl92c_bb_block_on(struct ieee80211_hw *hw); -u32 _rtl92c_phy_calculate_bit_shift(u32 bitmask); long _rtl92c_phy_txpwr_idx_to_dbm(struct ieee80211_hw *hw, enum wireless_mode wirelessmode, u8 txpwridx); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.c index da54e51badd3..fa70a7d5539f 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.c @@ -39,7 +39,7 @@ u32 rtl92c_phy_query_rf_reg(struct ieee80211_hw *hw, rfpath, regaddr); } - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -110,7 +110,7 @@ void rtl92ce_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = _rtl92c_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); @@ -122,7 +122,7 @@ void rtl92ce_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = _rtl92c_phy_fw_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.h index 7582a162bd11..c7a0d4c776f0 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/phy.h @@ -94,7 +94,6 @@ u32 _rtl92c_phy_rf_serial_read(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset); u32 _rtl92c_phy_fw_rf_serial_read(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset); -u32 _rtl92c_phy_calculate_bit_shift(u32 bitmask); void _rtl92c_phy_rf_serial_write(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset, u32 data); void _rtl92c_phy_fw_rf_serial_write(struct ieee80211_hw *hw, diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/phy.c index a8d9fe269f31..0b8cb7e61fd8 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/phy.c @@ -32,7 +32,7 @@ u32 rtl92cu_phy_query_rf_reg(struct ieee80211_hw *hw, original_value = _rtl92c_phy_fw_rf_serial_read(hw, rfpath, regaddr); } - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), rfpath(%#x), bitmask(%#x), original_value(%#x)\n", @@ -56,7 +56,7 @@ void rtl92cu_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = _rtl92c_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); @@ -67,7 +67,7 @@ void rtl92cu_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = _rtl92c_phy_fw_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92c_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c index d18c092b6142..d835a27429f0 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c @@ -169,13 +169,6 @@ static const u8 channel_all[59] = { 157, 159, 161, 163, 165 }; -static u32 _rtl92d_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} - u32 rtl92d_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) { struct rtl_priv *rtlpriv = rtl_priv(hw); @@ -198,7 +191,7 @@ u32 rtl92d_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) } else { originalvalue = rtl_read_dword(rtlpriv, regaddr); } - bitshift = _rtl92d_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "BBR MASK=0x%x Addr[0x%x]=0x%x\n", @@ -230,7 +223,7 @@ void rtl92d_phy_set_bb_reg(struct ieee80211_hw *hw, dbi_direct); else originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92d_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } if (rtlhal->during_mac1init_radioa || rtlhal->during_mac0init_radiob) @@ -317,7 +310,7 @@ u32 rtl92d_phy_query_rf_reg(struct ieee80211_hw *hw, regaddr, rfpath, bitmask); spin_lock(&rtlpriv->locks.rf_lock); original_value = _rtl92d_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92d_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -343,7 +336,7 @@ void rtl92d_phy_set_rf_reg(struct ieee80211_hw *hw, enum radio_path rfpath, if (bitmask != RFREG_OFFSET_MASK) { original_value = _rtl92d_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92d_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); } diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/phy.c index cc0bcaf13e96..73ef602bfb01 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/phy.c @@ -16,7 +16,6 @@ static u32 _rtl92ee_phy_rf_serial_read(struct ieee80211_hw *hw, static void _rtl92ee_phy_rf_serial_write(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset, u32 data); -static u32 _rtl92ee_phy_calculate_bit_shift(u32 bitmask); static bool _rtl92ee_phy_bb8192ee_config_parafile(struct ieee80211_hw *hw); static bool _rtl92ee_phy_config_mac_with_headerfile(struct ieee80211_hw *hw); static bool phy_config_bb_with_hdr_file(struct ieee80211_hw *hw, @@ -46,7 +45,7 @@ u32 rtl92ee_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), bitmask(%#x)\n", regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92ee_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -68,7 +67,7 @@ void rtl92ee_phy_set_bb_reg(struct ieee80211_hw *hw, u32 regaddr, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92ee_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } @@ -92,7 +91,7 @@ u32 rtl92ee_phy_query_rf_reg(struct ieee80211_hw *hw, spin_lock(&rtlpriv->locks.rf_lock); original_value = _rtl92ee_phy_rf_serial_read(hw , rfpath, regaddr); - bitshift = _rtl92ee_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -119,7 +118,7 @@ void rtl92ee_phy_set_rf_reg(struct ieee80211_hw *hw, if (bitmask != RFREG_OFFSET_MASK) { original_value = _rtl92ee_phy_rf_serial_read(hw, rfpath, addr); - bitshift = _rtl92ee_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = (original_value & (~bitmask)) | (data << bitshift); } @@ -201,13 +200,6 @@ static void _rtl92ee_phy_rf_serial_write(struct ieee80211_hw *hw, pphyreg->rf3wire_offset, data_and_addr); } -static u32 _rtl92ee_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} - bool rtl92ee_phy_mac_config(struct ieee80211_hw *hw) { return _rtl92ee_phy_config_mac_with_headerfile(hw); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192se/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192se/phy.c index 09591a0b5a81..d9ef7e1da1db 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192se/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192se/phy.c @@ -14,13 +14,6 @@ #include "hw.h" #include "table.h" -static u32 _rtl92s_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} - u32 rtl92s_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) { struct rtl_priv *rtlpriv = rtl_priv(hw); @@ -30,7 +23,7 @@ u32 rtl92s_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92s_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "BBR MASK=0x%x Addr[0x%x]=0x%x\n", @@ -52,7 +45,7 @@ void rtl92s_phy_set_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl92s_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } @@ -157,7 +150,7 @@ u32 rtl92s_phy_query_rf_reg(struct ieee80211_hw *hw, enum radio_path rfpath, original_value = _rtl92s_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92s_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -188,7 +181,7 @@ void rtl92s_phy_set_rf_reg(struct ieee80211_hw *hw, enum radio_path rfpath, if (bitmask != RFREG_OFFSET_MASK) { original_value = _rtl92s_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl92s_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); } diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8723ae/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8723ae/phy.c index fe9b407dc2af..71e29b103da5 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8723ae/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8723ae/phy.c @@ -49,7 +49,7 @@ u32 rtl8723e_phy_query_rf_reg(struct ieee80211_hw *hw, rfpath, regaddr); } - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -80,7 +80,7 @@ void rtl8723e_phy_set_rf_reg(struct ieee80211_hw *hw, original_value = rtl8723_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); @@ -89,7 +89,7 @@ void rtl8723e_phy_set_rf_reg(struct ieee80211_hw *hw, rtl8723_phy_rf_serial_write(hw, rfpath, regaddr, data); } else { if (bitmask != RFREG_OFFSET_MASK) { - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8723be/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8723be/phy.c index 2b9313cb93db..094cb36153f5 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8723be/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8723be/phy.c @@ -41,7 +41,7 @@ u32 rtl8723be_phy_query_rf_reg(struct ieee80211_hw *hw, enum radio_path rfpath, spin_lock(&rtlpriv->locks.rf_lock); original_value = rtl8723_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -68,7 +68,7 @@ void rtl8723be_phy_set_rf_reg(struct ieee80211_hw *hw, enum radio_path path, if (bitmask != RFREG_OFFSET_MASK) { original_value = rtl8723_phy_rf_serial_read(hw, path, regaddr); - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); } diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.c b/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.c index 47b6c1aa36b0..d97c88ebce75 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.c @@ -17,7 +17,7 @@ u32 rtl8723_phy_query_bb_reg(struct ieee80211_hw *hw, rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), bitmask(%#x)\n", regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -39,7 +39,7 @@ void rtl8723_phy_set_bb_reg(struct ieee80211_hw *hw, u32 regaddr, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = rtl8723_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | (data << bitshift)); } @@ -51,14 +51,6 @@ void rtl8723_phy_set_bb_reg(struct ieee80211_hw *hw, u32 regaddr, } EXPORT_SYMBOL_GPL(rtl8723_phy_set_bb_reg); -u32 rtl8723_phy_calculate_bit_shift(u32 bitmask) -{ - u32 i = ffs(bitmask); - - return i ? i - 1 : 32; -} -EXPORT_SYMBOL_GPL(rtl8723_phy_calculate_bit_shift); - u32 rtl8723_phy_rf_serial_read(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset) { diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.h b/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.h index edf1c52f0ee2..af85c3287507 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8723com/phy_common.h @@ -27,7 +27,6 @@ u32 rtl8723_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask); void rtl8723_phy_set_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask, u32 data); -u32 rtl8723_phy_calculate_bit_shift(u32 bitmask); u32 rtl8723_phy_rf_serial_read(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset); void rtl8723_phy_rf_serial_write(struct ieee80211_hw *hw, diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/phy.c index 68c3fb0395ce..1be51ea3f3c8 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/phy.c @@ -27,13 +27,6 @@ static u32 _rtl8821ae_phy_rf_serial_read(struct ieee80211_hw *hw, static void _rtl8821ae_phy_rf_serial_write(struct ieee80211_hw *hw, enum radio_path rfpath, u32 offset, u32 data); -static u32 _rtl8821ae_phy_calculate_bit_shift(u32 bitmask) -{ - if (WARN_ON_ONCE(!bitmask)) - return 0; - - return __ffs(bitmask); -} static bool _rtl8821ae_phy_bb8821a_config_parafile(struct ieee80211_hw *hw); /*static bool _rtl8812ae_phy_config_mac_with_headerfile(struct ieee80211_hw *hw);*/ static bool _rtl8821ae_phy_config_mac_with_headerfile(struct ieee80211_hw *hw); @@ -106,7 +99,7 @@ u32 rtl8821ae_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, "regaddr(%#x), bitmask(%#x)\n", regaddr, bitmask); originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl8821ae_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); returnvalue = (originalvalue & bitmask) >> bitshift; rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, @@ -127,7 +120,7 @@ void rtl8821ae_phy_set_bb_reg(struct ieee80211_hw *hw, if (bitmask != MASKDWORD) { originalvalue = rtl_read_dword(rtlpriv, regaddr); - bitshift = _rtl8821ae_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((originalvalue & (~bitmask)) | ((data << bitshift) & bitmask)); } @@ -153,7 +146,7 @@ u32 rtl8821ae_phy_query_rf_reg(struct ieee80211_hw *hw, spin_lock(&rtlpriv->locks.rf_lock); original_value = _rtl8821ae_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl8821ae_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); readback_value = (original_value & bitmask) >> bitshift; spin_unlock(&rtlpriv->locks.rf_lock); @@ -181,7 +174,7 @@ void rtl8821ae_phy_set_rf_reg(struct ieee80211_hw *hw, if (bitmask != RFREG_OFFSET_MASK) { original_value = _rtl8821ae_phy_rf_serial_read(hw, rfpath, regaddr); - bitshift = _rtl8821ae_phy_calculate_bit_shift(bitmask); + bitshift = calculate_bit_shift(bitmask); data = ((original_value & (~bitmask)) | (data << bitshift)); } diff --git a/drivers/net/wireless/realtek/rtlwifi/wifi.h b/drivers/net/wireless/realtek/rtlwifi/wifi.h index ac8dfda7099d..d87cd2252eac 100644 --- a/drivers/net/wireless/realtek/rtlwifi/wifi.h +++ b/drivers/net/wireless/realtek/rtlwifi/wifi.h @@ -3052,4 +3052,11 @@ static inline struct ieee80211_sta *rtl_find_sta(struct ieee80211_hw *hw, return ieee80211_find_sta(mac->vif, mac_addr); } +static inline u32 calculate_bit_shift(u32 bitmask) +{ + if (WARN_ON_ONCE(!bitmask)) + return 0; + + return __ffs(bitmask); +} #endif diff --git a/drivers/net/wireless/realtek/rtw88/fw.c b/drivers/net/wireless/realtek/rtw88/fw.c index acd78311c8c4..3f037ddcecf1 100644 --- a/drivers/net/wireless/realtek/rtw88/fw.c +++ b/drivers/net/wireless/realtek/rtw88/fw.c @@ -998,7 +998,7 @@ static u8 rtw_get_rsvd_page_probe_req_location(struct rtw_dev *rtwdev, if (rsvd_pkt->type != RSVD_PROBE_REQ) continue; if ((!ssid && !rsvd_pkt->ssid) || - rtw_ssid_equal(rsvd_pkt->ssid, ssid)) + cfg80211_ssid_eq(rsvd_pkt->ssid, ssid)) location = rsvd_pkt->page; } @@ -1015,7 +1015,7 @@ static u16 rtw_get_rsvd_page_probe_req_size(struct rtw_dev *rtwdev, if (rsvd_pkt->type != RSVD_PROBE_REQ) continue; if ((!ssid && !rsvd_pkt->ssid) || - rtw_ssid_equal(rsvd_pkt->ssid, ssid)) + cfg80211_ssid_eq(rsvd_pkt->ssid, ssid)) size = rsvd_pkt->probe_req_size; } diff --git a/drivers/net/wireless/realtek/rtw88/main.h b/drivers/net/wireless/realtek/rtw88/main.h index b6bfd4c02e2d..e14d1da43940 100644 --- a/drivers/net/wireless/realtek/rtw88/main.h +++ b/drivers/net/wireless/realtek/rtw88/main.h @@ -2090,18 +2090,6 @@ static inline struct ieee80211_vif *rtwvif_to_vif(struct rtw_vif *rtwvif) return container_of(p, struct ieee80211_vif, drv_priv); } -static inline bool rtw_ssid_equal(struct cfg80211_ssid *a, - struct cfg80211_ssid *b) -{ - if (!a || !b || a->ssid_len != b->ssid_len) - return false; - - if (memcmp(a->ssid, b->ssid, a->ssid_len)) - return false; - - return true; -} - static inline void rtw_chip_efuse_grant_on(struct rtw_dev *rtwdev) { if (rtwdev->chip->ops->efuse_grant) diff --git a/drivers/net/wireless/realtek/rtw89/coex.c b/drivers/net/wireless/realtek/rtw89/coex.c index 9c0db35d3e13..f37afb4cbb63 100644 --- a/drivers/net/wireless/realtek/rtw89/coex.c +++ b/drivers/net/wireless/realtek/rtw89/coex.c @@ -6,6 +6,7 @@ #include "debug.h" #include "fw.h" #include "mac.h" +#include "phy.h" #include "ps.h" #include "reg.h" @@ -122,7 +123,8 @@ static const u32 cxtbl[] = { 0xea55556a, /* 21 */ 0xaafafafa, /* 22 */ 0xfafaaafa, /* 23 */ - 0xfafffaff /* 24 */ + 0xfafffaff, /* 24 */ + 0xea6a5a5a, /* 25 */ }; static const struct rtw89_btc_ver rtw89_btc_ver_defs[] = { @@ -131,7 +133,7 @@ static const struct rtw89_btc_ver rtw89_btc_ver_defs[] = { .fcxbtcrpt = 105, .fcxtdma = 3, .fcxslots = 1, .fcxcysta = 5, .fcxstep = 3, .fcxnullsta = 2, .fcxmreg = 2, .fcxgpiodbg = 1, .fcxbtver = 1, .fcxbtscan = 2, .fcxbtafh = 2, .fcxbtdevinfo = 1, - .fwlrole = 1, .frptmap = 3, .fcxctrl = 1, + .fwlrole = 2, .frptmap = 3, .fcxctrl = 1, .info_buf = 1800, .max_role_num = 6, }, {RTL8852C, RTW89_FW_VER_CODE(0, 27, 57, 0), @@ -159,7 +161,7 @@ static const struct rtw89_btc_ver rtw89_btc_ver_defs[] = { .fcxbtcrpt = 105, .fcxtdma = 3, .fcxslots = 1, .fcxcysta = 5, .fcxstep = 3, .fcxnullsta = 2, .fcxmreg = 2, .fcxgpiodbg = 1, .fcxbtver = 1, .fcxbtscan = 2, .fcxbtafh = 2, .fcxbtdevinfo = 1, - .fwlrole = 1, .frptmap = 3, .fcxctrl = 1, + .fwlrole = 2, .frptmap = 3, .fcxctrl = 1, .info_buf = 1800, .max_role_num = 6, }, {RTL8852B, RTW89_FW_VER_CODE(0, 29, 14, 0), @@ -246,6 +248,11 @@ struct rtw89_btc_btf_set_mon_reg { struct rtw89_btc_fbtc_mreg regs[] __counted_by(reg_num); } __packed; +struct _wl_rinfo_now { + u8 link_mode; + u32 dbcc_2g_phy: 2; +}; + enum btc_btf_set_cx_policy { CXPOLICY_TDMA = 0x0, CXPOLICY_SLOT = 0x1, @@ -262,6 +269,8 @@ enum btc_b2w_scoreboard { BTC_BSCB_RFK_RUN = BIT(5), BTC_BSCB_RFK_REQ = BIT(6), BTC_BSCB_LPS = BIT(7), + BTC_BSCB_BT_LNAB0 = BIT(8), + BTC_BSCB_BT_LNAB1 = BIT(10), BTC_BSCB_WLRFK = BIT(11), BTC_BSCB_BT_HILNA = BIT(13), BTC_BSCB_BT_CONNECT = BIT(16), @@ -405,11 +414,14 @@ enum btc_cx_poicy_type { /* TDMA Fix slot-8: W1:B1 = user-define */ BTC_CXP_FIX_TDW1B1 = (BTC_CXP_FIX << 8) | 8, - /* TDMA Fix slot-9: W1:B1 = 40:20 */ - BTC_CXP_FIX_TD4020 = (BTC_CXP_FIX << 8) | 9, - /* TDMA Fix slot-9: W1:B1 = 40:10 */ - BTC_CXP_FIX_TD4010ISO = (BTC_CXP_FIX << 8) | 10, + BTC_CXP_FIX_TD4010ISO = (BTC_CXP_FIX << 8) | 9, + + /* TDMA Fix slot-10: W1:B1 = 40:10 */ + BTC_CXP_FIX_TD4010ISO_DL = (BTC_CXP_FIX << 8) | 10, + + /* TDMA Fix slot-11: W1:B1 = 40:10 */ + BTC_CXP_FIX_TD4010ISO_UL = (BTC_CXP_FIX << 8) | 11, /* PS-TDMA Fix slot-0: W1:B1 = 30:30 */ BTC_CXP_PFIX_TD3030 = (BTC_CXP_PFIX << 8) | 0, @@ -710,7 +722,8 @@ static void _reset_btc_var(struct rtw89_dev *rtwdev, u8 type) if (type & BTC_RESET_CX) memset(cx, 0, sizeof(*cx)); - else if (type & BTC_RESET_BTINFO) /* only for BT enable */ + + if (type & BTC_RESET_BTINFO) /* only for BT enable */ memset(bt, 0, sizeof(*bt)); if (type & BTC_RESET_CTRL) { @@ -739,12 +752,115 @@ static void _reset_btc_var(struct rtw89_dev *rtwdev, u8 type) btc->dm.coex_info_map = BTC_COEX_INFO_ALL; btc->dm.wl_tx_limit.tx_time = BTC_MAX_TX_TIME_DEF; btc->dm.wl_tx_limit.tx_retry = BTC_MAX_TX_RETRY_DEF; + btc->dm.wl_pre_agc_rb = BTC_PREAGC_NOTFOUND; + btc->dm.wl_btg_rx_rb = BTC_BTGCTRL_BB_GNT_NOTFOUND; } if (type & BTC_RESET_MDINFO) memset(&btc->mdinfo, 0, sizeof(btc->mdinfo)); } +static u8 _search_reg_index(struct rtw89_dev *rtwdev, u8 mreg_num, u16 reg_type, u32 target) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + u8 i; + + for (i = 0; i < mreg_num; i++) + if (le16_to_cpu(chip->mon_reg[i].type) == reg_type && + le32_to_cpu(chip->mon_reg[i].offset) == target) { + return i; + } + return BTC_REG_NOTFOUND; +} + +static void _get_reg_status(struct rtw89_dev *rtwdev, u8 type, u8 *val) +{ + struct rtw89_btc *btc = &rtwdev->btc; + const struct rtw89_btc_ver *ver = btc->ver; + struct rtw89_btc_module *md = &btc->mdinfo; + union rtw89_btc_fbtc_mreg_val *pmreg; + u32 pre_agc_addr = R_BTC_BB_PRE_AGC_S1; + u32 reg_val; + u8 idx; + + if (md->ant.btg_pos == RF_PATH_A) + pre_agc_addr = R_BTC_BB_PRE_AGC_S0; + + switch (type) { + case BTC_CSTATUS_TXDIV_POS: + if (md->switch_type == BTC_SWITCH_INTERNAL) + *val = BTC_ANT_DIV_MAIN; + break; + case BTC_CSTATUS_RXDIV_POS: + if (md->switch_type == BTC_SWITCH_INTERNAL) + *val = BTC_ANT_DIV_MAIN; + break; + case BTC_CSTATUS_BB_GNT_MUX: + reg_val = rtw89_phy_read32(rtwdev, R_BTC_BB_BTG_RX); + *val = !(reg_val & B_BTC_BB_GNT_MUX); + break; + case BTC_CSTATUS_BB_GNT_MUX_MON: + if (!btc->fwinfo.rpt_fbtc_mregval.cinfo.valid) + return; + + pmreg = &btc->fwinfo.rpt_fbtc_mregval.finfo; + if (ver->fcxmreg == 1) { + idx = _search_reg_index(rtwdev, pmreg->v1.reg_num, + REG_BB, R_BTC_BB_BTG_RX); + if (idx == BTC_REG_NOTFOUND) { + *val = BTC_BTGCTRL_BB_GNT_NOTFOUND; + } else { + reg_val = le32_to_cpu(pmreg->v1.mreg_val[idx]); + *val = !(reg_val & B_BTC_BB_GNT_MUX); + } + } else if (ver->fcxmreg == 2) { + idx = _search_reg_index(rtwdev, pmreg->v2.reg_num, + REG_BB, R_BTC_BB_BTG_RX); + if (idx == BTC_REG_NOTFOUND) { + *val = BTC_BTGCTRL_BB_GNT_NOTFOUND; + } else { + reg_val = le32_to_cpu(pmreg->v2.mreg_val[idx]); + *val = !(reg_val & B_BTC_BB_GNT_MUX); + } + } + break; + case BTC_CSTATUS_BB_PRE_AGC: + reg_val = rtw89_phy_read32(rtwdev, pre_agc_addr); + reg_val &= B_BTC_BB_PRE_AGC_MASK; + *val = (reg_val == B_BTC_BB_PRE_AGC_VAL); + break; + case BTC_CSTATUS_BB_PRE_AGC_MON: + if (!btc->fwinfo.rpt_fbtc_mregval.cinfo.valid) + return; + + pmreg = &btc->fwinfo.rpt_fbtc_mregval.finfo; + if (ver->fcxmreg == 1) { + idx = _search_reg_index(rtwdev, pmreg->v1.reg_num, + REG_BB, pre_agc_addr); + if (idx == BTC_REG_NOTFOUND) { + *val = BTC_PREAGC_NOTFOUND; + } else { + reg_val = le32_to_cpu(pmreg->v1.mreg_val[idx]) & + B_BTC_BB_PRE_AGC_MASK; + *val = (reg_val == B_BTC_BB_PRE_AGC_VAL); + } + } else if (ver->fcxmreg == 2) { + idx = _search_reg_index(rtwdev, pmreg->v2.reg_num, + REG_BB, pre_agc_addr); + if (idx == BTC_REG_NOTFOUND) { + *val = BTC_PREAGC_NOTFOUND; + } else { + reg_val = le32_to_cpu(pmreg->v2.mreg_val[idx]) & + B_BTC_BB_PRE_AGC_MASK; + *val = (reg_val == B_BTC_BB_PRE_AGC_VAL); + } + } + break; + default: + break; + } +} + #define BTC_RPT_HDR_SIZE 3 #define BTC_CHK_WLSLOT_DRIFT_MAX 15 #define BTC_CHK_BTSLOT_DRIFT_MAX 15 @@ -1003,7 +1119,7 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, u16 wl_slot_set = 0, wl_slot_real = 0; u32 trace_step = btc->ctrl.trace_step, rpt_len = 0, diff_t = 0; u32 cnt_leak_slot, bt_slot_real, bt_slot_set, cnt_rx_imr; - u8 i; + u8 i, val = 0; rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): index:%d\n", @@ -1508,6 +1624,19 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, goto err; } break; + case BTC_RPT_TYPE_MREG: + _get_reg_status(rtwdev, BTC_CSTATUS_BB_GNT_MUX_MON, &val); + if (dm->wl_btg_rx == BTC_BTGCTRL_BB_GNT_FWCTRL) + dm->wl_btg_rx_rb = BTC_BTGCTRL_BB_GNT_FWCTRL; + else + dm->wl_btg_rx_rb = val; + + _get_reg_status(rtwdev, BTC_CSTATUS_BB_PRE_AGC_MON, &val); + if (dm->wl_pre_agc == BTC_PREAGC_BB_FWCTRL) + dm->wl_pre_agc_rb = BTC_PREAGC_BB_FWCTRL; + else + dm->wl_pre_agc_rb = val; + break; case BTC_RPT_TYPE_BT_VER: case BTC_RPT_TYPE_BT_SCAN: case BTC_RPT_TYPE_BT_AFH: @@ -2155,8 +2284,9 @@ static void _set_bt_rx_gain(struct rtw89_dev *rtwdev, u8 level) struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_bt_info *bt = &btc->cx.bt; - if (bt->rf_para.rx_gain_freerun == level || - level > BTC_BT_RX_NORMAL_LVL) + if ((bt->rf_para.rx_gain_freerun == level || + level > BTC_BT_RX_NORMAL_LVL) && + (!rtwdev->chip->scbd || bt->lna_constrain == level)) return; bt->rf_para.rx_gain_freerun = level; @@ -2171,32 +2301,59 @@ static void _set_bt_rx_gain(struct rtw89_dev *rtwdev, u8 level) else _write_scbd(rtwdev, BTC_WSCB_RXGAIN, true); - _send_fw_cmd(rtwdev, BTFC_SET, SET_BT_LNA_CONSTRAIN, &level, 1); + _send_fw_cmd(rtwdev, BTFC_SET, SET_BT_LNA_CONSTRAIN, &level, sizeof(level)); } static void _set_rf_trx_para(struct rtw89_dev *rtwdev) { const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; + const struct rtw89_btc_ver *ver = btc->ver; struct rtw89_btc_dm *dm = &btc->dm; struct rtw89_btc_wl_info *wl = &btc->cx.wl; struct rtw89_btc_bt_info *bt = &btc->cx.bt; struct rtw89_btc_bt_link_info *b = &bt->link_info; + struct rtw89_btc_wl_smap *wl_smap = &wl->status.map; struct rtw89_btc_rf_trx_para para; u32 wl_stb_chg = 0; - u8 level_id = 0; + u8 level_id = 0, link_mode = 0, i, dbcc_2g_phy = 0; + + if (ver->fwlrole == 0) { + link_mode = wl->role_info.link_mode; + for (i = 0; i < RTW89_PHY_MAX; i++) { + if (wl->dbcc_info.real_band[i] == RTW89_BAND_2G) + dbcc_2g_phy = i; + } + } else if (ver->fwlrole == 1) { + link_mode = wl->role_info_v1.link_mode; + dbcc_2g_phy = wl->role_info_v1.dbcc_2g_phy; + } else if (ver->fwlrole == 2) { + link_mode = wl->role_info_v2.link_mode; + dbcc_2g_phy = wl->role_info_v2.dbcc_2g_phy; + } - if (!dm->freerun) { - /* fix LNA2 = level-5 for BT ACI issue at BTG */ + /* decide trx_para_level */ + if (btc->mdinfo.ant.type == BTC_ANT_SHARED) { + /* fix LNA2 + TIA gain not change by GNT_BT */ if ((btc->dm.wl_btg_rx && b->profile_cnt.now != 0) || dm->bt_only == 1) - dm->trx_para_level = 1; + dm->trx_para_level = 1; /* for better BT ACI issue */ else dm->trx_para_level = 0; + } else { /* non-shared antenna */ + dm->trx_para_level = 5; + /* modify trx_para if WK 2.4G-STA-DL + bt link */ + if (b->profile_cnt.now != 0 && + link_mode == BTC_WLINK_2G_STA && + wl->status.map.traffic_dir & BIT(RTW89_TFC_UL)) { /* uplink */ + if (wl->rssi_level == 4 && bt->rssi_level > 2) + dm->trx_para_level = 6; + else if (wl->rssi_level == 3 && bt->rssi_level > 3) + dm->trx_para_level = 7; + } } - level_id = (u8)dm->trx_para_level; - + level_id = dm->trx_para_level; if (level_id >= chip->rf_para_dlink_num || level_id >= chip->rf_para_ulink_num) { rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -2210,25 +2367,26 @@ static void _set_rf_trx_para(struct rtw89_dev *rtwdev) else para = chip->rf_para_dlink[level_id]; - if (para.wl_tx_power != RTW89_BTC_WL_DEF_TX_PWR) - rtw89_debug(rtwdev, RTW89_DBG_BTC, - "[BTC], %s(): wl_tx_power=%d\n", - __func__, para.wl_tx_power); - _set_wl_tx_power(rtwdev, para.wl_tx_power); - _set_wl_rx_gain(rtwdev, para.wl_rx_gain); - _set_bt_tx_power(rtwdev, para.bt_tx_power); - _set_bt_rx_gain(rtwdev, para.bt_rx_gain); - - if (bt->enable.now == 0 || wl->status.map.rf_off == 1 || - wl->status.map.lps == BTC_LPS_RF_OFF) + if (dm->fddt_train) { + _set_wl_rx_gain(rtwdev, 1); + _write_scbd(rtwdev, BTC_WSCB_RXGAIN, true); + } else { + _set_wl_tx_power(rtwdev, para.wl_tx_power); + _set_wl_rx_gain(rtwdev, para.wl_rx_gain); + _set_bt_tx_power(rtwdev, para.bt_tx_power); + _set_bt_rx_gain(rtwdev, para.bt_rx_gain); + } + + if (!bt->enable.now || dm->wl_only || wl_smap->rf_off || + wl_smap->lps == BTC_LPS_RF_OFF || + link_mode == BTC_WLINK_5G || + link_mode == BTC_WLINK_NOLINK || + (rtwdev->dbcc_en && dbcc_2g_phy != RTW89_PHY_1)) wl_stb_chg = 0; else wl_stb_chg = 1; if (wl_stb_chg != dm->wl_stb_chg) { - rtw89_debug(rtwdev, RTW89_DBG_BTC, - "[BTC], %s(): wl_stb_chg=%d\n", - __func__, wl_stb_chg); dm->wl_stb_chg = wl_stb_chg; chip->ops->btc_wl_s1_standby(rtwdev, dm->wl_stb_chg); } @@ -2661,9 +2819,17 @@ void rtw89_btc_set_policy(struct rtw89_dev *rtwdev, u16 policy_type) _slot_set(btc, CXST_W1, 40, tbl_w1, SLOT_ISO); _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); break; - case BTC_CXP_FIX_TD4020: - _slot_set(btc, CXST_W1, 40, cxtbl[1], SLOT_MIX); - _slot_set(btc, CXST_B1, 20, tbl_b1, SLOT_MIX); + case BTC_CXP_FIX_TD4010ISO: + _slot_set(btc, CXST_W1, 40, cxtbl[1], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD4010ISO_DL: + _slot_set(btc, CXST_W1, 40, cxtbl[25], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, cxtbl[25], SLOT_ISO); + break; + case BTC_CXP_FIX_TD4010ISO_UL: + _slot_set(btc, CXST_W1, 40, cxtbl[20], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, cxtbl[25], SLOT_MIX); break; case BTC_CXP_FIX_TD7010: _slot_set(btc, CXST_W1, 70, tbl_w1, SLOT_ISO); @@ -3002,9 +3168,13 @@ void rtw89_btc_set_policy_v1(struct rtw89_dev *rtwdev, u16 policy_type) _slot_set(btc, CXST_W1, 40, cxtbl[1], SLOT_ISO); _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); break; - case BTC_CXP_FIX_TD4020: - _slot_set(btc, CXST_W1, 40, cxtbl[1], SLOT_MIX); - _slot_set(btc, CXST_B1, 20, tbl_b1, SLOT_MIX); + case BTC_CXP_FIX_TD4010ISO_DL: + _slot_set(btc, CXST_W1, 40, cxtbl[25], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, cxtbl[25], SLOT_ISO); + break; + case BTC_CXP_FIX_TD4010ISO_UL: + _slot_set(btc, CXST_W1, 40, cxtbl[20], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, cxtbl[25], SLOT_MIX); break; case BTC_CXP_FIX_TD7010: _slot_set(btc, CXST_W1, 70, tbl_w1, SLOT_ISO); @@ -3381,17 +3551,32 @@ static void _action_wl_init(struct rtw89_dev *rtwdev) _set_policy(rtwdev, BTC_CXP_OFF_BT, BTC_ACT_WL_INIT); } -static void _action_wl_off(struct rtw89_dev *rtwdev) +static void _action_wl_off(struct rtw89_dev *rtwdev, u8 mode) { struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): !!\n", __func__); - if (wl->status.map.rf_off || btc->dm.bt_only) + if (wl->status.map.rf_off || btc->dm.bt_only) { _set_ant(rtwdev, NM_EXEC, BTC_PHY_ALL, BTC_ANT_WOFF); + } else if (wl->status.map.lps == BTC_LPS_RF_ON) { + if (wl->role_info.link_mode == BTC_WLINK_5G) + _set_ant(rtwdev, FC_EXEC, BTC_PHY_ALL, BTC_ANT_W5G); + else + _set_ant(rtwdev, FC_EXEC, BTC_PHY_ALL, BTC_ANT_W2G); + } - _set_policy(rtwdev, BTC_CXP_OFF_BT, BTC_ACT_WL_OFF); + if (mode == BTC_WLINK_5G) { + _set_policy(rtwdev, BTC_CXP_OFF_EQ0, BTC_ACT_WL_OFF); + } else if (wl->status.map.lps == BTC_LPS_RF_ON) { + if (btc->cx.bt.link_info.a2dp_desc.active) + _set_policy(rtwdev, BTC_CXP_OFF_BT, BTC_ACT_WL_OFF); + else + _set_policy(rtwdev, BTC_CXP_OFF_BWB1, BTC_ACT_WL_OFF); + } else { + _set_policy(rtwdev, BTC_CXP_OFF_BT, BTC_ACT_WL_OFF); + } } static void _action_freerun(struct rtw89_dev *rtwdev) @@ -3426,31 +3611,25 @@ static void _action_bt_idle(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_bt_link_info *b = &btc->cx.bt.link_info; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; _set_ant(rtwdev, NM_EXEC, BTC_PHY_ALL, BTC_ANT_W2G); if (btc->mdinfo.ant.type == BTC_ANT_SHARED) { /* shared-antenna */ switch (btc->cx.state_map) { case BTC_WBUSY_BNOSCAN: /*wl-busy + bt idle*/ - if (b->profile_cnt.now > 0) - _set_policy(rtwdev, BTC_CXP_FIX_TD4010, - BTC_ACT_BT_IDLE); + case BTC_WSCAN_BNOSCAN: /* wl-scan + bt-idle */ + if (b->status.map.connect) + _set_policy(rtwdev, BTC_CXP_FIX_TD4010, BTC_ACT_BT_IDLE); + else if (wl->status.map.traffic_dir & BIT(RTW89_TFC_DL)) + _set_policy(rtwdev, BTC_CXP_FIX_TD4010ISO_DL, BTC_ACT_BT_IDLE); else - _set_policy(rtwdev, BTC_CXP_FIX_TD4020, - BTC_ACT_BT_IDLE); + _set_policy(rtwdev, BTC_CXP_FIX_TD4010ISO_UL, BTC_ACT_BT_IDLE); break; case BTC_WBUSY_BSCAN: /*wl-busy + bt-inq */ _set_policy(rtwdev, BTC_CXP_PFIX_TD5050, BTC_ACT_BT_IDLE); break; - case BTC_WSCAN_BNOSCAN: /* wl-scan + bt-idle */ - if (b->profile_cnt.now > 0) - _set_policy(rtwdev, BTC_CXP_FIX_TD4010, - BTC_ACT_BT_IDLE); - else - _set_policy(rtwdev, BTC_CXP_FIX_TD4020, - BTC_ACT_BT_IDLE); - break; case BTC_WSCAN_BSCAN: /* wl-scan + bt-inq */ _set_policy(rtwdev, BTC_CXP_FIX_TD5050, BTC_ACT_BT_IDLE); @@ -3617,7 +3796,7 @@ static void _action_bt_pan(struct rtw89_dev *rtwdev) _set_policy(rtwdev, BTC_CXP_FIX_TD3060, BTC_ACT_BT_PAN); break; case BTC_WLINKING: /* wl-connecting + bt-PAN */ - _set_policy(rtwdev, BTC_CXP_FIX_TD4020, BTC_ACT_BT_PAN); + _set_policy(rtwdev, BTC_CXP_FIX_TD4010ISO, BTC_ACT_BT_PAN); break; case BTC_WIDLE: /* wl-idle + bt-pan */ _set_policy(rtwdev, BTC_CXP_PFIX_TD2080, BTC_ACT_BT_PAN); @@ -3798,46 +3977,134 @@ static void _action_wl_rfk(struct rtw89_dev *rtwdev) static void _set_btg_ctrl(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; - const struct rtw89_btc_ver *ver = btc->ver; struct rtw89_btc_wl_info *wl = &btc->cx.wl; - struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; struct rtw89_btc_wl_role_info_v2 *wl_rinfo_v2 = &wl->role_info_v2; + struct rtw89_btc_wl_role_info *wl_rinfo_v0 = &wl->role_info; struct rtw89_btc_wl_dbcc_info *wl_dinfo = &wl->dbcc_info; - bool is_btg; - u8 mode; + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_btc_ver *ver = btc->ver; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + struct rtw89_btc_dm *dm = &btc->dm; + struct _wl_rinfo_now wl_rinfo; + u32 run_reason = btc->dm.run_reason; + u32 is_btg; + u8 i, val; if (btc->ctrl.manual) return; if (ver->fwlrole == 0) - mode = wl_rinfo->link_mode; + wl_rinfo.link_mode = wl_rinfo_v0->link_mode; else if (ver->fwlrole == 1) - mode = wl_rinfo_v1->link_mode; + wl_rinfo.link_mode = wl_rinfo_v1->link_mode; else if (ver->fwlrole == 2) - mode = wl_rinfo_v2->link_mode; + wl_rinfo.link_mode = wl_rinfo_v2->link_mode; else return; - /* notify halbb ignore GNT_BT or not for WL BB Rx-AGC control */ - if (mode == BTC_WLINK_5G) /* always 0 if 5G */ - is_btg = false; - else if (mode == BTC_WLINK_25G_DBCC && - wl_dinfo->real_band[RTW89_PHY_1] != RTW89_BAND_2G) - is_btg = false; + if (rtwdev->dbcc_en) { + if (ver->fwlrole == 0) { + for (i = 0; i < RTW89_PHY_MAX; i++) { + if (wl_dinfo->real_band[i] == RTW89_BAND_2G) + wl_rinfo.dbcc_2g_phy = i; + } + } else if (ver->fwlrole == 1) { + wl_rinfo.dbcc_2g_phy = wl_rinfo_v1->dbcc_2g_phy; + } else if (ver->fwlrole == 2) { + wl_rinfo.dbcc_2g_phy = wl_rinfo_v2->dbcc_2g_phy; + } else { + return; + } + } + + if (wl_rinfo.link_mode == BTC_WLINK_25G_MCC) + is_btg = BTC_BTGCTRL_BB_GNT_FWCTRL; + else if (!(bt->run_patch_code && bt->enable.now)) + is_btg = BTC_BTGCTRL_DISABLE; + else if (wl_rinfo.link_mode == BTC_WLINK_5G) + is_btg = BTC_BTGCTRL_DISABLE; + else if (dm->freerun) + is_btg = BTC_BTGCTRL_DISABLE; + else if (rtwdev->dbcc_en && wl_rinfo.dbcc_2g_phy != RTW89_PHY_1) + is_btg = BTC_BTGCTRL_DISABLE; else - is_btg = true; + is_btg = BTC_BTGCTRL_ENABLE; - if (btc->dm.run_reason != BTC_RSN_NTFY_INIT && - is_btg == btc->dm.wl_btg_rx) - return; + if (dm->wl_btg_rx_rb != dm->wl_btg_rx && + dm->wl_btg_rx_rb != BTC_BTGCTRL_BB_GNT_NOTFOUND) { + _get_reg_status(rtwdev, BTC_CSTATUS_BB_GNT_MUX, &val); + dm->wl_btg_rx_rb = val; + } + + if (run_reason == BTC_RSN_NTFY_INIT || + run_reason == BTC_RSN_NTFY_SWBAND || + dm->wl_btg_rx_rb != dm->wl_btg_rx || + is_btg != dm->wl_btg_rx) { + + dm->wl_btg_rx = is_btg; - btc->dm.wl_btg_rx = is_btg; + if (is_btg > BTC_BTGCTRL_ENABLE) + return; + + chip->ops->ctrl_btg_bt_rx(rtwdev, is_btg, RTW89_PHY_0); + } +} - if (mode == BTC_WLINK_25G_MCC) +static void _set_wl_preagc_ctrl(struct rtw89_dev *rtwdev) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_bt_link_info *bt_linfo = &btc->cx.bt.link_info; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_wl_role_info_v2 *wl_rinfo = &wl->role_info_v2; + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_btc_ver *ver = btc->ver; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + struct rtw89_btc_dm *dm = &btc->dm; + u8 is_preagc, val; + + if (btc->ctrl.manual) return; - rtw89_ctrl_btg_bt_rx(rtwdev, is_btg, RTW89_PHY_0); + if (wl_rinfo->link_mode == BTC_WLINK_25G_MCC) + is_preagc = BTC_PREAGC_BB_FWCTRL; + else if (!(bt->run_patch_code && bt->enable.now)) + is_preagc = BTC_PREAGC_DISABLE; + else if (wl_rinfo->link_mode == BTC_WLINK_5G) + is_preagc = BTC_PREAGC_DISABLE; + else if (wl_rinfo->link_mode == BTC_WLINK_NOLINK || + btc->cx.bt.link_info.profile_cnt.now == 0) + is_preagc = BTC_PREAGC_DISABLE; + else if (dm->tdma_now.type != CXTDMA_OFF && + !bt_linfo->hfp_desc.exist && + !bt_linfo->hid_desc.exist && + dm->fddt_train == BTC_FDDT_DISABLE) + is_preagc = BTC_PREAGC_DISABLE; + else if (ver->fwlrole == 2 && wl_rinfo->dbcc_en && + wl_rinfo->dbcc_2g_phy != RTW89_PHY_1) + is_preagc = BTC_PREAGC_DISABLE; + else if (btc->mdinfo.ant.type == BTC_ANT_SHARED) + is_preagc = BTC_PREAGC_DISABLE; + else + is_preagc = BTC_PREAGC_ENABLE; + + if (dm->wl_pre_agc_rb != dm->wl_pre_agc && + dm->wl_pre_agc_rb != BTC_PREAGC_NOTFOUND) { + _get_reg_status(rtwdev, BTC_CSTATUS_BB_PRE_AGC, &val); + dm->wl_pre_agc_rb = val; + } + + if ((wl->coex_mode == BTC_MODE_NORMAL && + (dm->run_reason == BTC_RSN_NTFY_INIT || + dm->run_reason == BTC_RSN_NTFY_SWBAND || + dm->wl_pre_agc_rb != dm->wl_pre_agc)) || + is_preagc != dm->wl_pre_agc) { + dm->wl_pre_agc = is_preagc; + + if (is_preagc > BTC_PREAGC_ENABLE) + return; + chip->ops->ctrl_nbtg_bt_tx(rtwdev, dm->wl_pre_agc, RTW89_PHY_0); + } } struct rtw89_txtime_data { @@ -4024,6 +4291,7 @@ static void _action_common(struct rtw89_dev *rtwdev) struct rtw89_btc_wl_info *wl = &btc->cx.wl; _set_btg_ctrl(rtwdev); + _set_wl_preagc_ctrl(rtwdev); _set_wl_tx_limit(rtwdev); _set_bt_afh_info(rtwdev); _set_bt_rx_agc(rtwdev); @@ -5008,8 +5276,7 @@ static void _update_bt_scbd(struct rtw89_dev *rtwdev, bool only_update) return; } - if (!(val & BTC_BSCB_ON) || - btc->dm.cnt_dm[BTC_DCNT_BTCNT_HANG] >= BTC_CHK_HANG_MAX) + if (!(val & BTC_BSCB_ON)) bt->enable.now = 0; else bt->enable.now = 1; @@ -5035,6 +5302,9 @@ static void _update_bt_scbd(struct rtw89_dev *rtwdev, bool only_update) bt->btg_type = val & BTC_BSCB_BT_S1 ? BTC_BT_BTG : BTC_BT_ALONE; bt->link_info.a2dp_desc.exist = !!(val & BTC_BSCB_A2DP_ACT); + bt->lna_constrain = !!(val & BTC_BSCB_BT_LNAB0) + + !!(val & BTC_BSCB_BT_LNAB1) * 2 + 4; + /* if rfk run 1->0 */ if (bt->rfk_info.map.run && !(val & BTC_BSCB_RFK_RUN)) status_change = true; @@ -5128,17 +5398,28 @@ void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) } if (wl->status.map.rf_off_pre == wl->status.map.rf_off && - wl->status.map.lps_pre == wl->status.map.lps && - (reason == BTC_RSN_NTFY_POWEROFF || - reason == BTC_RSN_NTFY_RADIO_STATE)) { - rtw89_debug(rtwdev, RTW89_DBG_BTC, - "[BTC], %s(): return for WL rf off state no change!!\n", - __func__); - return; + wl->status.map.lps_pre == wl->status.map.lps) { + if (reason == BTC_RSN_NTFY_POWEROFF || + reason == BTC_RSN_NTFY_RADIO_STATE) { + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], %s(): return for WL rf off state no change!!\n", + __func__); + return; + } + if (wl->status.map.rf_off == 1 || + wl->status.map.lps == BTC_LPS_RF_OFF) { + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], %s(): return for WL rf off state!!\n", + __func__); + return; + } } + dm->freerun = false; dm->cnt_dm[BTC_DCNT_RUN]++; dm->fddt_train = BTC_FDDT_DISABLE; + btc->ctrl.igno_bt = false; + bt->scan_rx_low_pri = false; if (btc->ctrl.always_freerun) { _action_freerun(rtwdev); @@ -5153,15 +5434,11 @@ void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) } if (wl->status.map.rf_off || wl->status.map.lps || dm->bt_only) { - _action_wl_off(rtwdev); + _action_wl_off(rtwdev, mode); btc->ctrl.igno_bt = true; goto exit; } - btc->ctrl.igno_bt = false; - dm->freerun = false; - bt->scan_rx_low_pri = false; - if (reason == BTC_RSN_NTFY_INIT) { _action_wl_init(rtwdev); goto exit; @@ -5186,12 +5463,14 @@ void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) if (mode == BTC_WLINK_NOLINK || mode == BTC_WLINK_2G_STA || mode == BTC_WLINK_5G) { _action_wl_scan(rtwdev); + bt->scan_rx_low_pri = false; goto exit; } } if (wl->status.map.scan) { _action_wl_scan(rtwdev); + bt->scan_rx_low_pri = false; goto exit; } @@ -5308,6 +5587,7 @@ void rtw89_btc_ntfy_init(struct rtw89_dev *rtwdev, u8 mode) rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): mode=%d\n", __func__, mode); + wl->coex_mode = mode; dm->cnt_notify[BTC_NCNT_INIT_COEX]++; dm->wl_only = mode == BTC_MODE_WL ? 1 : 0; dm->bt_only = mode == BTC_MODE_BT ? 1 : 0; @@ -5525,6 +5805,37 @@ void rtw89_btc_ntfy_icmp_packet_work(struct work_struct *work) mutex_unlock(&rtwdev->mutex); } +static u8 _update_bt_rssi_level(struct rtw89_dev *rtwdev, u8 rssi) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + u8 *rssi_st, rssi_th, rssi_level = 0; + u8 i; + + /* for rssi locate in which {40, 36, 31, 28} + * if rssi >= 40% (-60dBm) --> rssi_level = 4 + * if 36% <= rssi < 40% --> rssi_level = 3 + * if 31% <= rssi < 36% --> rssi_level = 2 + * if 28% <= rssi < 31% --> rssi_level = 1 + * if rssi < 28% --> rssi_level = 0 + */ + + /* check if rssi across bt_rssi_thres boundary */ + for (i = 0; i < BTC_BT_RSSI_THMAX; i++) { + rssi_th = chip->bt_rssi_thres[i]; + rssi_st = &bt->link_info.rssi_state[i]; + + *rssi_st = _update_rssi_state(rtwdev, *rssi_st, rssi, rssi_th); + + if (BTC_RSSI_HIGH(*rssi_st)) { + rssi_level = BTC_BT_RSSI_THMAX - i; + break; + } + } + return rssi_level; +} + #define BT_PROFILE_PROTOCOL_MASK GENMASK(7, 4) static void _update_bt_info(struct rtw89_dev *rtwdev, u8 *buf, u32 len) @@ -5600,7 +5911,8 @@ static void _update_bt_info(struct rtw89_dev *rtwdev, u8 *buf, u32 len) btinfo.val = bt->raw_info[BTC_BTINFO_H0]; /* raw val is dBm unit, translate from -100~ 0dBm to 0~100%*/ b->rssi = chip->ops->btc_get_bt_rssi(rtwdev, btinfo.hb0.rssi); - btc->dm.trx_info.bt_rssi = b->rssi; + bt->rssi_level = _update_bt_rssi_level(rtwdev, b->rssi); + btc->dm.trx_info.bt_rssi = bt->rssi_level; /* parse raw info high-Byte1 */ btinfo.val = bt->raw_info[BTC_BTINFO_H1]; @@ -5804,22 +6116,22 @@ void rtw89_btc_ntfy_radio_state(struct rtw89_dev *rtwdev, enum btc_rfctrl rf_sta chip->ops->btc_init_cfg(rtwdev); } else { rtw89_btc_fw_en_rpt(rtwdev, RPT_EN_ALL, false); - if (rf_state == BTC_RFCTRL_WL_OFF) + if (rf_state == BTC_RFCTRL_FW_CTRL) + _write_scbd(rtwdev, BTC_WSCB_ACTIVE, false); + else if (rf_state == BTC_RFCTRL_WL_OFF) _write_scbd(rtwdev, BTC_WSCB_ALL, false); - else if (rf_state == BTC_RFCTRL_LPS_WL_ON && - wl->status.map.lps_pre != BTC_LPS_OFF) + else + _write_scbd(rtwdev, BTC_WSCB_ACTIVE, false); + + if (rf_state == BTC_RFCTRL_LPS_WL_ON && + wl->status.map.lps_pre != BTC_LPS_OFF) _update_bt_scbd(rtwdev, true); } btc->dm.cnt_dm[BTC_DCNT_BTCNT_HANG] = 0; - if (wl->status.map.lps_pre == BTC_LPS_OFF && - wl->status.map.lps_pre != wl->status.map.lps) - btc->dm.tdma_instant_excute = 1; - else - btc->dm.tdma_instant_excute = 0; + btc->dm.tdma_instant_excute = 1; _run_coex(rtwdev, BTC_RSN_NTFY_RADIO_STATE); - btc->dm.tdma_instant_excute = 0; wl->status.map.rf_off_pre = wl->status.map.rf_off; wl->status.map.lps_pre = wl->status.map.lps; } @@ -6058,6 +6370,13 @@ static void rtw89_btc_ntfy_wl_sta_iter(void *data, struct ieee80211_sta *sta) dm->trx_info.tx_tp = link_info_t->tx_throughput; dm->trx_info.rx_tp = link_info_t->rx_throughput; + /* Trigger coex-run if 0x10980 reg-value is diff with coex setup */ + if ((dm->wl_btg_rx_rb != dm->wl_btg_rx && + dm->wl_btg_rx_rb != BTC_BTGCTRL_BB_GNT_NOTFOUND) || + (dm->wl_pre_agc_rb != dm->wl_pre_agc && + dm->wl_pre_agc_rb != BTC_PREAGC_NOTFOUND)) + iter_data->is_sta_change = true; + if (is_sta_change) iter_data->is_sta_change = true; @@ -6443,8 +6762,9 @@ static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m) bt_linfo->pan_desc.active ? "Y" : "N"); seq_printf(m, - " %-15s : rssi:%ddBm, tx_rate:%dM, %s%s%s", + " %-15s : rssi:%ddBm(lvl:%d), tx_rate:%dM, %s%s%s", "[link]", bt_linfo->rssi - 100, + bt->rssi_level, bt_linfo->tx_3m ? 3 : 2, bt_linfo->status.map.inq_pag ? " inq-page!!" : "", bt_linfo->status.map.acl_busy ? " acl_busy!!" : "", @@ -6553,6 +6873,8 @@ static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m) case BTC_CXP_ ## e | BTC_POLICY_EXT_BIT: return #e #define CASE_BTC_SLOT_STR(e) case CXST_ ## e: return #e #define CASE_BTC_EVT_STR(e) case CXEVNT_## e: return #e +#define CASE_BTC_INIT(e) case BTC_MODE_## e: return #e +#define CASE_BTC_ANTPATH_STR(e) case BTC_ANT_##e: return #e static const char *steps_to_str(u16 step) { @@ -6633,8 +6955,9 @@ static const char *steps_to_str(u16 step) CASE_BTC_POLICY_STR(FIX_TD3060); CASE_BTC_POLICY_STR(FIX_TD2080); CASE_BTC_POLICY_STR(FIX_TDW1B1); - CASE_BTC_POLICY_STR(FIX_TD4020); CASE_BTC_POLICY_STR(FIX_TD4010ISO); + CASE_BTC_POLICY_STR(FIX_TD4010ISO_DL); + CASE_BTC_POLICY_STR(FIX_TD4010ISO_UL); CASE_BTC_POLICY_STR(PFIX_TD3030); CASE_BTC_POLICY_STR(PFIX_TD5050); CASE_BTC_POLICY_STR(PFIX_TD2030); @@ -6727,6 +7050,37 @@ static const char *id_to_evt(u32 id) } } +static const char *id_to_mode(u8 id) +{ + switch (id) { + CASE_BTC_INIT(NORMAL); + CASE_BTC_INIT(WL); + CASE_BTC_INIT(BT); + CASE_BTC_INIT(WLOFF); + default: + return "unknown"; + } +} + +static const char *id_to_ant(u32 id) +{ + switch (id) { + CASE_BTC_ANTPATH_STR(WPOWERON); + CASE_BTC_ANTPATH_STR(WINIT); + CASE_BTC_ANTPATH_STR(WONLY); + CASE_BTC_ANTPATH_STR(WOFF); + CASE_BTC_ANTPATH_STR(W2G); + CASE_BTC_ANTPATH_STR(W5G); + CASE_BTC_ANTPATH_STR(W25G); + CASE_BTC_ANTPATH_STR(FREERUN); + CASE_BTC_ANTPATH_STR(WRFK); + CASE_BTC_ANTPATH_STR(BRFK); + CASE_BTC_ANTPATH_STR(MAX); + default: + return "unknown"; + } +} + static void seq_print_segment(struct seq_file *m, const char *prefix, u16 *data, u8 len, u8 seg_len, u8 start_idx, u8 ring_len) @@ -6781,12 +7135,13 @@ static void _show_dm_info(struct rtw89_dev *rtwdev, struct seq_file *m) (btc->ctrl.manual ? "(Manual)" : "(Auto)")); seq_printf(m, - " %-15s : type:%s, reason:%s(), action:%s(), ant_path:%ld, run_cnt:%d\n", + " %-15s : type:%s, reason:%s(), action:%s(), ant_path:%s, init_mode:%s, run_cnt:%d\n", "[status]", module->ant.type == BTC_ANT_SHARED ? "shared" : "dedicated", steps_to_str(dm->run_reason), steps_to_str(dm->run_action | BTC_ACT_EXT_BIT), - FIELD_GET(GENMASK(7, 0), dm->set_ant_path), + id_to_ant(FIELD_GET(GENMASK(7, 0), dm->set_ant_path)), + id_to_mode(wl->coex_mode), dm->cnt_dm[BTC_DCNT_RUN]); _show_dm_step(rtwdev, m); @@ -7689,7 +8044,8 @@ static void _get_gnt(struct rtw89_dev *rtwdev, struct rtw89_mac_ax_coex_gnt *gnt struct rtw89_mac_ax_gnt *gnt; u32 val, status; - if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B) { + if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B || + chip->chip_id == RTL8851B) { rtw89_mac_read_lte(rtwdev, R_AX_LTE_SW_CFG_1, &val); rtw89_mac_read_lte(rtwdev, R_AX_GNT_VAL, &status); @@ -7751,27 +8107,25 @@ static void _show_mreg_v1(struct rtw89_dev *rtwdev, struct seq_file *m) bt->scbd, cx->cnt_bt[BTC_BCNT_SCBDREAD], cx->cnt_bt[BTC_BCNT_SCBDUPDATE]); - /* To avoid I/O if WL LPS or power-off */ - if (!wl->status.map.lps && !wl->status.map.rf_off) { - btc->dm.pta_owner = rtw89_mac_get_ctrl_path(rtwdev); + btc->dm.pta_owner = rtw89_mac_get_ctrl_path(rtwdev); + _get_gnt(rtwdev, &gnt_cfg); + + gnt = gnt_cfg.band[0]; + seq_printf(m, + " %-15s : pta_owner:%s, phy-0[gnt_wl:%s-%d/gnt_bt:%s-%d], ", + "[gnt_status]", + chip->chip_id == RTL8852C ? "HW" : + btc->dm.pta_owner == BTC_CTRL_BY_WL ? "WL" : "BT", + gnt.gnt_wl_sw_en ? "SW" : "HW", gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", gnt.gnt_bt); + + gnt = gnt_cfg.band[1]; + seq_printf(m, "phy-1[gnt_wl:%s-%d/gnt_bt:%s-%d]\n", + gnt.gnt_wl_sw_en ? "SW" : "HW", + gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", + gnt.gnt_bt); - _get_gnt(rtwdev, &gnt_cfg); - gnt = gnt_cfg.band[0]; - seq_printf(m, - " %-15s : pta_owner:%s, phy-0[gnt_wl:%s-%d/gnt_bt:%s-%d], ", - "[gnt_status]", - chip->chip_id == RTL8852C ? "HW" : - btc->dm.pta_owner == BTC_CTRL_BY_WL ? "WL" : "BT", - gnt.gnt_wl_sw_en ? "SW" : "HW", gnt.gnt_wl, - gnt.gnt_bt_sw_en ? "SW" : "HW", gnt.gnt_bt); - - gnt = gnt_cfg.band[1]; - seq_printf(m, "phy-1[gnt_wl:%s-%d/gnt_bt:%s-%d]\n", - gnt.gnt_wl_sw_en ? "SW" : "HW", - gnt.gnt_wl, - gnt.gnt_bt_sw_en ? "SW" : "HW", - gnt.gnt_bt); - } pcinfo = &pfwinfo->rpt_fbtc_mregval.cinfo; if (!pcinfo->valid) { rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -7855,27 +8209,25 @@ static void _show_mreg_v2(struct rtw89_dev *rtwdev, struct seq_file *m) bt->scbd, cx->cnt_bt[BTC_BCNT_SCBDREAD], cx->cnt_bt[BTC_BCNT_SCBDUPDATE]); - /* To avoid I/O if WL LPS or power-off */ - if (!wl->status.map.lps && !wl->status.map.rf_off) { - btc->dm.pta_owner = rtw89_mac_get_ctrl_path(rtwdev); + btc->dm.pta_owner = rtw89_mac_get_ctrl_path(rtwdev); + _get_gnt(rtwdev, &gnt_cfg); + + gnt = gnt_cfg.band[0]; + seq_printf(m, + " %-15s : pta_owner:%s, phy-0[gnt_wl:%s-%d/gnt_bt:%s-%d], ", + "[gnt_status]", + chip->chip_id == RTL8852C ? "HW" : + btc->dm.pta_owner == BTC_CTRL_BY_WL ? "WL" : "BT", + gnt.gnt_wl_sw_en ? "SW" : "HW", gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", gnt.gnt_bt); + + gnt = gnt_cfg.band[1]; + seq_printf(m, "phy-1[gnt_wl:%s-%d/gnt_bt:%s-%d]\n", + gnt.gnt_wl_sw_en ? "SW" : "HW", + gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", + gnt.gnt_bt); - _get_gnt(rtwdev, &gnt_cfg); - gnt = gnt_cfg.band[0]; - seq_printf(m, - " %-15s : pta_owner:%s, phy-0[gnt_wl:%s-%d/gnt_bt:%s-%d], ", - "[gnt_status]", - chip->chip_id == RTL8852C ? "HW" : - btc->dm.pta_owner == BTC_CTRL_BY_WL ? "WL" : "BT", - gnt.gnt_wl_sw_en ? "SW" : "HW", gnt.gnt_wl, - gnt.gnt_bt_sw_en ? "SW" : "HW", gnt.gnt_bt); - - gnt = gnt_cfg.band[1]; - seq_printf(m, "phy-1[gnt_wl:%s-%d/gnt_bt:%s-%d]\n", - gnt.gnt_wl_sw_en ? "SW" : "HW", - gnt.gnt_wl, - gnt.gnt_bt_sw_en ? "SW" : "HW", - gnt.gnt_bt); - } pcinfo = &pfwinfo->rpt_fbtc_mregval.cinfo; if (!pcinfo->valid) { rtw89_debug(rtwdev, RTW89_DBG_BTC, diff --git a/drivers/net/wireless/realtek/rtw89/coex.h b/drivers/net/wireless/realtek/rtw89/coex.h index e76153709793..46e25c6f88a6 100644 --- a/drivers/net/wireless/realtek/rtw89/coex.h +++ b/drivers/net/wireless/realtek/rtw89/coex.h @@ -142,6 +142,44 @@ enum btc_lps_state { BTC_LPS_RF_ON = 2 }; +#define R_BTC_BB_BTG_RX 0x980 +#define R_BTC_BB_PRE_AGC_S1 0x476C +#define R_BTC_BB_PRE_AGC_S0 0x4688 + +#define B_BTC_BB_GNT_MUX GENMASK(20, 17) +#define B_BTC_BB_PRE_AGC_MASK GENMASK(31, 24) +#define B_BTC_BB_PRE_AGC_VAL BIT(31) + +#define BTC_REG_NOTFOUND 0xff + +enum btc_ant_div_pos { + BTC_ANT_DIV_MAIN = 0, + BTC_ANT_DIV_AUX = 1, +}; + +enum btc_get_reg_status { + BTC_CSTATUS_TXDIV_POS = 0, + BTC_CSTATUS_RXDIV_POS = 1, + BTC_CSTATUS_BB_GNT_MUX = 2, + BTC_CSTATUS_BB_GNT_MUX_MON = 3, + BTC_CSTATUS_BB_PRE_AGC = 4, + BTC_CSTATUS_BB_PRE_AGC_MON = 5, +}; + +enum btc_preagc_type { + BTC_PREAGC_DISABLE, + BTC_PREAGC_ENABLE, + BTC_PREAGC_BB_FWCTRL, + BTC_PREAGC_NOTFOUND, +}; + +enum btc_btgctrl_type { + BTC_BTGCTRL_DISABLE, + BTC_BTGCTRL_ENABLE, + BTC_BTGCTRL_BB_GNT_FWCTRL, + BTC_BTGCTRL_BB_GNT_NOTFOUND, +}; + void rtw89_btc_ntfy_poweron(struct rtw89_dev *rtwdev); void rtw89_btc_ntfy_poweroff(struct rtw89_dev *rtwdev); void rtw89_btc_ntfy_init(struct rtw89_dev *rtwdev, u8 mode); diff --git a/drivers/net/wireless/realtek/rtw89/core.h b/drivers/net/wireless/realtek/rtw89/core.h index 21421980fd06..ea6df859ba15 100644 --- a/drivers/net/wireless/realtek/rtw89/core.h +++ b/drivers/net/wireless/realtek/rtw89/core.h @@ -1706,6 +1706,7 @@ struct rtw89_btc_wl_info { u8 port_id[RTW89_WIFI_ROLE_MLME_MAX]; u8 rssi_level; u8 cn_report; + u8 coex_mode; bool scbd_change; u32 scbd; @@ -1813,6 +1814,7 @@ struct rtw89_btc_bt_info { union rtw89_btc_bt_rfk_info_map rfk_info; u8 raw_info[BTC_BTINFO_MAX]; /* raw bt info from mailbox */ + u8 rssi_level; u32 scbd; u32 feature; @@ -1829,7 +1831,8 @@ struct rtw89_btc_bt_info { u32 hi_lna_rx: 1; u32 scan_rx_low_pri: 1; u32 scan_info_update: 1; - u32 rsvd: 20; + u32 lna_constrain: 3; + u32 rsvd: 17; }; struct rtw89_btc_cx { @@ -2307,12 +2310,6 @@ struct rtw89_btc_fbtc_fddt_cell_status { u8 state_phase; /* [0:3] train state, [4:7] train phase */ } __packed; -struct rtw89_btc_fbtc_fddt_cell_status_v5 { - s8 wl_tx_pwr; - s8 bt_tx_pwr; - s8 bt_rx_gain; -} __packed; - struct rtw89_btc_fbtc_cysta_v3 { /* statistics for cycles */ u8 fver; u8 rsvd; @@ -2376,9 +2373,9 @@ struct rtw89_btc_fbtc_cysta_v5 { /* statistics for cycles */ struct rtw89_btc_fbtc_cycle_a2dp_empty_info a2dp_ept; struct rtw89_btc_fbtc_a2dp_trx_stat_v4 a2dp_trx[BTC_CYCLE_SLOT_MAX]; struct rtw89_btc_fbtc_cycle_fddt_info_v5 fddt_trx[BTC_CYCLE_SLOT_MAX]; - struct rtw89_btc_fbtc_fddt_cell_status_v5 fddt_cells[FDD_TRAIN_WL_DIRECTION] - [FDD_TRAIN_WL_RSSI_LEVEL] - [FDD_TRAIN_BT_RSSI_LEVEL]; + struct rtw89_btc_fbtc_fddt_cell_status fddt_cells[FDD_TRAIN_WL_DIRECTION] + [FDD_TRAIN_WL_RSSI_LEVEL] + [FDD_TRAIN_BT_RSSI_LEVEL]; __le32 except_map; } __packed; @@ -2511,18 +2508,22 @@ struct rtw89_btc_dm { u32 noisy_level: 3; u32 coex_info_map: 8; u32 bt_only: 1; - u32 wl_btg_rx: 1; + u32 wl_btg_rx: 2; u32 trx_para_level: 8; u32 wl_stb_chg: 1; u32 pta_owner: 1; + u32 tdma_instant_excute: 1; + u32 wl_btg_rx_rb: 2; u16 slot_dur[CXST_MAX]; u8 run_reason; u8 run_action; + u8 wl_pre_agc: 2; u8 wl_lna2: 1; + u8 wl_pre_agc_rb: 2; }; struct rtw89_btc_ctrl { diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c index 80ab5fdfdaa0..c485ef2cc3d3 100644 --- a/drivers/net/wireless/realtek/rtw89/mac.c +++ b/drivers/net/wireless/realtek/rtw89/mac.c @@ -5507,7 +5507,8 @@ bool rtw89_mac_get_ctrl_path(struct rtw89_dev *rtwdev) if (chip->chip_id == RTL8852C) return false; - else if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B) + else if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B || + chip->chip_id == RTL8851B) val = rtw89_read8_mask(rtwdev, R_AX_SYS_SDIO_CTRL + 3, B_AX_LTE_MUX_CTRL_PATH >> 24); diff --git a/drivers/net/wireless/realtek/rtw89/mac_be.c b/drivers/net/wireless/realtek/rtw89/mac_be.c index fa3f5ef289cb..be30c9346293 100644 --- a/drivers/net/wireless/realtek/rtw89/mac_be.c +++ b/drivers/net/wireless/realtek/rtw89/mac_be.c @@ -73,6 +73,11 @@ static int rtw89_mac_check_mac_en_be(struct rtw89_dev *rtwdev, u8 mac_idx, return -EFAULT; } +static bool is_qta_poh(struct rtw89_dev *rtwdev) +{ + return rtwdev->hci.type == RTW89_HCI_TYPE_PCIE; +} + static void hfc_get_mix_info_be(struct rtw89_dev *rtwdev) { struct rtw89_hfc_param *param = &rtwdev->mac.hfc_param; @@ -566,6 +571,800 @@ static int rtw89_fwdl_check_path_ready_be(struct rtw89_dev *rtwdev, rtwdev, R_BE_WCPU_FW_CTRL); } +static int dmac_func_en_be(struct rtw89_dev *rtwdev) +{ + return 0; +} + +static int cmac_func_en_be(struct rtw89_dev *rtwdev, u8 mac_idx, bool en) +{ + u32 reg; + + if (mac_idx > RTW89_MAC_1) + return -EINVAL; + + if (mac_idx == RTW89_MAC_0) + return 0; + + if (en) { + rtw89_write32_set(rtwdev, R_BE_AFE_CTRL1, B_BE_AFE_CTRL1_SET); + rtw89_write32_clr(rtwdev, R_BE_SYS_ISO_CTRL_EXTEND, B_BE_R_SYM_ISO_CMAC12PP); + rtw89_write32_set(rtwdev, R_BE_FEN_RST_ENABLE, B_BE_CMAC1_FEN); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CK_EN, mac_idx); + rtw89_write32_set(rtwdev, reg, B_BE_CK_EN_SET); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CMAC_FUNC_EN, mac_idx); + rtw89_write32_set(rtwdev, reg, B_BE_CMAC_FUNC_EN_SET); + + set_bit(RTW89_FLAG_CMAC1_FUNC, rtwdev->flags); + } else { + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CMAC_FUNC_EN, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_CMAC_FUNC_EN_SET); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CK_EN, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_CK_EN_SET); + + rtw89_write32_clr(rtwdev, R_BE_FEN_RST_ENABLE, B_BE_CMAC1_FEN); + rtw89_write32_set(rtwdev, R_BE_SYS_ISO_CTRL_EXTEND, B_BE_R_SYM_ISO_CMAC12PP); + rtw89_write32_clr(rtwdev, R_BE_AFE_CTRL1, B_BE_AFE_CTRL1_SET); + + clear_bit(RTW89_FLAG_CMAC1_FUNC, rtwdev->flags); + } + + return 0; +} + +static int chip_func_en_be(struct rtw89_dev *rtwdev) +{ + return 0; +} + +static int sys_init_be(struct rtw89_dev *rtwdev) +{ + int ret; + + ret = dmac_func_en_be(rtwdev); + if (ret) + return ret; + + ret = cmac_func_en_be(rtwdev, RTW89_MAC_0, true); + if (ret) + return ret; + + ret = chip_func_en_be(rtwdev); + if (ret) + return ret; + + return ret; +} + +static int sta_sch_init_be(struct rtw89_dev *rtwdev) +{ + u32 p_val; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, RTW89_MAC_0, RTW89_DMAC_SEL); + if (ret) + return ret; + + rtw89_write8_set(rtwdev, R_BE_SS_CTRL, B_BE_SS_EN); + + ret = read_poll_timeout(rtw89_read32, p_val, p_val & B_BE_SS_INIT_DONE, + 1, TRXCFG_WAIT_CNT, false, rtwdev, R_BE_SS_CTRL); + if (ret) { + rtw89_err(rtwdev, "[ERR]STA scheduler init\n"); + return ret; + } + + rtw89_write32_set(rtwdev, R_BE_SS_CTRL, B_BE_WARM_INIT); + rtw89_write32_clr(rtwdev, R_BE_SS_CTRL, B_BE_BAND_TRIG_EN | B_BE_BAND1_TRIG_EN); + + return 0; +} + +static int mpdu_proc_init_be(struct rtw89_dev *rtwdev) +{ + u32 val32; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, RTW89_MAC_0, RTW89_DMAC_SEL); + if (ret) + return ret; + + rtw89_write32_set(rtwdev, R_BE_MPDU_PROC, B_BE_APPEND_FCS); + rtw89_write32(rtwdev, R_BE_CUT_AMSDU_CTRL, TRXCFG_MPDU_PROC_CUT_CTRL); + + val32 = rtw89_read32(rtwdev, R_BE_HDR_SHCUT_SETTING); + val32 |= (B_BE_TX_HW_SEQ_EN | B_BE_TX_HW_ACK_POLICY_EN | B_BE_TX_MAC_MPDU_PROC_EN); + val32 &= ~B_BE_TX_ADDR_MLD_TO_LIK; + rtw89_write32_set(rtwdev, R_BE_HDR_SHCUT_SETTING, val32); + + rtw89_write32(rtwdev, R_BE_RX_HDRTRNS, TRXCFG_MPDU_PROC_RX_HDR_CONV); + + val32 = rtw89_read32(rtwdev, R_BE_DISP_FWD_WLAN_0); + val32 = u32_replace_bits(val32, 1, B_BE_FWD_WLAN_CPU_TYPE_0_DATA_MASK); + val32 = u32_replace_bits(val32, 1, B_BE_FWD_WLAN_CPU_TYPE_0_MNG_MASK); + val32 = u32_replace_bits(val32, 1, B_BE_FWD_WLAN_CPU_TYPE_0_CTL_MASK); + val32 = u32_replace_bits(val32, 1, B_BE_FWD_WLAN_CPU_TYPE_1_MASK); + rtw89_write32(rtwdev, R_BE_DISP_FWD_WLAN_0, val32); + + return 0; +} + +static int sec_eng_init_be(struct rtw89_dev *rtwdev) +{ + u32 val32; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, RTW89_MAC_0, RTW89_DMAC_SEL); + if (ret) + return ret; + + val32 = rtw89_read32(rtwdev, R_BE_SEC_ENG_CTRL); + val32 |= B_BE_CLK_EN_CGCMP | B_BE_CLK_EN_WAPI | B_BE_CLK_EN_WEP_TKIP | + B_BE_SEC_TX_ENC | B_BE_SEC_RX_DEC | + B_BE_MC_DEC | B_BE_BC_DEC | + B_BE_BMC_MGNT_DEC | B_BE_UC_MGNT_DEC; + val32 &= ~B_BE_SEC_PRE_ENQUE_TX; + rtw89_write32(rtwdev, R_BE_SEC_ENG_CTRL, val32); + + rtw89_write32_set(rtwdev, R_BE_SEC_MPDU_PROC, B_BE_APPEND_ICV | B_BE_APPEND_MIC); + + return 0; +} + +static int txpktctrl_init_be(struct rtw89_dev *rtwdev) +{ + struct rtw89_mac_dle_rsvd_qt_cfg qt_cfg; + u32 val32; + int ret; + + ret = rtw89_mac_get_dle_rsvd_qt_cfg(rtwdev, DLE_RSVD_QT_MPDU_INFO, &qt_cfg); + if (ret) { + rtw89_err(rtwdev, "get dle rsvd qt %d cfg fail %d\n", + DLE_RSVD_QT_MPDU_INFO, ret); + return ret; + } + + val32 = rtw89_read32(rtwdev, R_BE_TXPKTCTL_MPDUINFO_CFG); + val32 = u32_replace_bits(val32, qt_cfg.pktid, B_BE_MPDUINFO_PKTID_MASK); + val32 = u32_replace_bits(val32, MPDU_INFO_B1_OFST, B_BE_MPDUINFO_B1_BADDR_MASK); + val32 |= B_BE_MPDUINFO_FEN; + rtw89_write32(rtwdev, R_BE_TXPKTCTL_MPDUINFO_CFG, val32); + + return 0; +} + +static int mlo_init_be(struct rtw89_dev *rtwdev) +{ + u32 val32; + int ret; + + val32 = rtw89_read32(rtwdev, R_BE_MLO_INIT_CTL); + + val32 |= B_BE_MLO_TABLE_REINIT; + rtw89_write32(rtwdev, R_BE_MLO_INIT_CTL, val32); + val32 &= ~B_BE_MLO_TABLE_REINIT; + rtw89_write32(rtwdev, R_BE_MLO_INIT_CTL, val32); + + ret = read_poll_timeout_atomic(rtw89_read32, val32, + val32 & B_BE_MLO_TABLE_INIT_DONE, + 1, 1000, false, rtwdev, R_BE_MLO_INIT_CTL); + if (ret) + rtw89_err(rtwdev, "[MLO]%s: MLO init polling timeout\n", __func__); + + rtw89_write32_set(rtwdev, R_BE_SS_CTRL, B_BE_MLO_HW_CHGLINK_EN); + rtw89_write32_set(rtwdev, R_BE_CMAC_SHARE_ACQCHK_CFG_0, B_BE_R_MACID_ACQ_CHK_EN); + + return ret; +} + +static int dmac_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + int ret; + + ret = rtw89_mac_dle_init(rtwdev, rtwdev->mac.qta_mode, RTW89_QTA_INVALID); + if (ret) { + rtw89_err(rtwdev, "[ERR]DLE init %d\n", ret); + return ret; + } + + ret = rtw89_mac_preload_init(rtwdev, RTW89_MAC_0, rtwdev->mac.qta_mode); + if (ret) { + rtw89_err(rtwdev, "[ERR]preload init %d\n", ret); + return ret; + } + + ret = rtw89_mac_hfc_init(rtwdev, true, true, true); + if (ret) { + rtw89_err(rtwdev, "[ERR]HCI FC init %d\n", ret); + return ret; + } + + ret = sta_sch_init_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR]STA SCH init %d\n", ret); + return ret; + } + + ret = mpdu_proc_init_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR]MPDU Proc init %d\n", ret); + return ret; + } + + ret = sec_eng_init_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR]Security Engine init %d\n", ret); + return ret; + } + + ret = txpktctrl_init_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR]TX pkt ctrl init %d\n", ret); + return ret; + } + + ret = mlo_init_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR]MLO init %d\n", ret); + return ret; + } + + return ret; +} + +static int scheduler_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_HE_CTN_CHK_CCA_NAV, mac_idx); + val32 = B_BE_HE_CTN_CHK_CCA_P20 | B_BE_HE_CTN_CHK_EDCCA_P20 | + B_BE_HE_CTN_CHK_CCA_BITMAP | B_BE_HE_CTN_CHK_EDCCA_BITMAP | + B_BE_HE_CTN_CHK_NO_GNT_WL | B_BE_HE_CTN_CHK_BASIC_NAV | + B_BE_HE_CTN_CHK_INTRA_NAV | B_BE_HE_CTN_CHK_TX_NAV; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_HE_SIFS_CHK_CCA_NAV, mac_idx); + val32 = B_BE_HE_SIFS_CHK_EDCCA_P20 | B_BE_HE_SIFS_CHK_EDCCA_BITMAP | + B_BE_HE_SIFS_CHK_NO_GNT_WL; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_TB_CHK_CCA_NAV, mac_idx); + val32 = B_BE_TB_CHK_EDCCA_BITMAP | B_BE_TB_CHK_NO_GNT_WL | B_BE_TB_CHK_BASIC_NAV; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CCA_CFG_0, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_NO_GNT_WL_EN); + + if (is_qta_poh(rtwdev)) { + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PREBKF_CFG_0, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_PREBKF_TIME_MASK, + SCH_PREBKF_24US); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CTN_CFG_0, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_PREBKF_TIME_NONAC_MASK, + SCH_PREBKF_24US); + } + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_EDCA_BCNQ_PARAM, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_BCNQ_CW_MASK, 0x32); + rtw89_write32_mask(rtwdev, reg, B_BE_BCNQ_AIFS_MASK, BCN_IFS_25US); + + return 0; +} + +static int addr_cam_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + u16 val16; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_ADDR_CAM_CTRL, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, ADDR_CAM_SERCH_RANGE, B_BE_ADDR_CAM_RANGE_MASK); + val32 |= B_BE_ADDR_CAM_EN; + if (mac_idx == RTW89_MAC_0) + val32 |= B_BE_ADDR_CAM_CLR; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_ADDR_CAM_CTRL, mac_idx); + ret = read_poll_timeout_atomic(rtw89_read16, val16, !(val16 & B_BE_ADDR_CAM_CLR), + 1, TRXCFG_WAIT_CNT, false, rtwdev, reg); + if (ret) + rtw89_err(rtwdev, "[ERR]ADDR_CAM reset\n"); + + return ret; +} + +static int rtw89_mac_typ_fltr_opt_be(struct rtw89_dev *rtwdev, + enum rtw89_machdr_frame_type type, + enum rtw89_mac_fwd_target fwd_target, + u8 mac_idx) +{ + u32 reg; + u32 val; + + switch (fwd_target) { + case RTW89_FWD_DONT_CARE: + val = RX_FLTR_FRAME_DROP_BE; + break; + case RTW89_FWD_TO_HOST: + case RTW89_FWD_TO_WLAN_CPU: + val = RX_FLTR_FRAME_ACCEPT_BE; + break; + default: + rtw89_err(rtwdev, "[ERR]set rx filter fwd target err\n"); + return -EINVAL; + } + + switch (type) { + case RTW89_MGNT: + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_MGNT_FLTR, mac_idx); + break; + case RTW89_CTRL: + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_CTRL_FLTR, mac_idx); + break; + case RTW89_DATA: + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_DATA_FLTR, mac_idx); + break; + default: + rtw89_err(rtwdev, "[ERR]set rx filter type err\n"); + return -EINVAL; + } + rtw89_write32(rtwdev, reg, val); + + return 0; +} + +static int rx_fltr_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 reg; + u32 val; + + rtw89_mac_typ_fltr_opt_be(rtwdev, RTW89_MGNT, RTW89_FWD_TO_HOST, mac_idx); + rtw89_mac_typ_fltr_opt_be(rtwdev, RTW89_CTRL, RTW89_FWD_TO_HOST, mac_idx); + rtw89_mac_typ_fltr_opt_be(rtwdev, RTW89_DATA, RTW89_FWD_TO_HOST, mac_idx); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RX_FLTR_OPT, mac_idx); + val = B_BE_A_BC_CAM_MATCH | B_BE_A_UC_CAM_MATCH | B_BE_A_MC | + B_BE_A_BC | B_BE_A_A1_MATCH | B_BE_SNIFFER_MODE | + u32_encode_bits(15, B_BE_UID_FILTER_MASK); + rtw89_write32(rtwdev, reg, val); + u32p_replace_bits(&rtwdev->hal.rx_fltr, 15, B_BE_UID_FILTER_MASK); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PLCP_HDR_FLTR, mac_idx); + val = B_BE_HE_SIGB_CRC_CHK | B_BE_VHT_MU_SIGB_CRC_CHK | + B_BE_VHT_SU_SIGB_CRC_CHK | B_BE_SIGA_CRC_CHK | + B_BE_LSIG_PARITY_CHK_EN | B_BE_CCK_SIG_CHK | B_BE_CCK_CRC_CHK; + rtw89_write16(rtwdev, reg, val); + + return 0; +} + +static int cca_ctrl_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + return 0; +} + +static int nav_ctrl_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + u32 reg; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_WMAC_NAV_CTL, mac_idx); + + val32 = rtw89_read32(rtwdev, reg); + val32 &= ~B_BE_WMAC_PLCP_UP_NAV_EN; + val32 |= B_BE_WMAC_TF_UP_NAV_EN | B_BE_WMAC_NAV_UPPER_EN; + val32 = u32_replace_bits(val32, NAV_25MS, B_BE_WMAC_NAV_UPPER_MASK); + + rtw89_write32(rtwdev, reg, val32); + + return 0; +} + +static int spatial_reuse_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RX_SR_CTRL, mac_idx); + rtw89_write8_clr(rtwdev, reg, B_BE_SR_EN | B_BE_SR_CTRL_PLCP_EN); + + return 0; +} + +static int tmac_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 reg; + + rtw89_write32_clr(rtwdev, R_BE_TB_PPDU_CTRL, B_BE_QOSNULL_UPD_MUEDCA_EN); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_WMTX_TCR_BE_4, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_EHT_HE_PPDU_4XLTF_ZLD_USTIMER_MASK, 0x12); + rtw89_write32_mask(rtwdev, reg, B_BE_EHT_HE_PPDU_2XLTF_ZLD_USTIMER_MASK, 0xe); + + return 0; +} + +static int trxptcl_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_rrsr_cfgs *rrsr = chip->rrsr_cfgs; + struct rtw89_hal *hal = &rtwdev->hal; + u32 val32; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_MAC_LOOPBACK, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, S_BE_MACLBK_PLCP_DLY_DEF, + B_BE_MACLBK_PLCP_DLY_MASK); + val32 &= ~B_BE_MACLBK_EN; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_TRXPTCL_RESP_0, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, WMAC_SPEC_SIFS_CCK, + B_BE_WMAC_SPEC_SIFS_CCK_MASK); + val32 = u32_replace_bits(val32, WMAC_SPEC_SIFS_OFDM_1115E, + B_BE_WMAC_SPEC_SIFS_OFDM_MASK); + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_WMAC_ACK_BA_RESP_LEGACY, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_ACK_BA_RESP_LEGACY_CHK_EDCCA); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_WMAC_ACK_BA_RESP_HE, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_ACK_BA_RESP_HE_CHK_EDCCA); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_WMAC_ACK_BA_RESP_EHT_LEG_PUNC, mac_idx); + rtw89_write32_clr(rtwdev, reg, B_BE_ACK_BA_EHT_LEG_PUNC_CHK_EDCCA); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RXTRIG_TEST_USER_2, mac_idx); + rtw89_write32_set(rtwdev, reg, B_BE_RXTRIG_FCSCHK_EN); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_TRXPTCL_RESP_1, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 &= B_BE_FTM_RRSR_RATE_EN_MASK | B_BE_WMAC_RESP_DOPPLEB_BE_EN | + B_BE_WMAC_RESP_DCM_EN | B_BE_WMAC_RESP_REF_RATE_MASK; + rtw89_write32(rtwdev, reg, val32); + rtw89_write32_mask(rtwdev, reg, rrsr->ref_rate.mask, rrsr->ref_rate.data); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_RRSR1, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 &= B_BE_RRSR_RATE_EN_MASK | B_BE_RRSR_CCK_MASK | B_BE_RSC_MASK; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_RRSR0, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 &= B_BE_RRSR_OFDM_MASK | B_BE_RRSR_HT_MASK | B_BE_RRSR_VHT_MASK | + B_BE_RRSR_HE_MASK; + rtw89_write32(rtwdev, reg, val32); + + if (chip->chip_id == RTL8922A && hal->cv == CHIP_CAV) { + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_RRSR1, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_RSC_MASK, 1); + } + + return 0; +} + +static int rst_bacam_be(struct rtw89_dev *rtwdev) +{ + u32 val; + int ret; + + rtw89_write32_mask(rtwdev, R_BE_RESPBA_CAM_CTRL, B_BE_BACAM_RST_MASK, + S_BE_BACAM_RST_ALL); + + ret = read_poll_timeout_atomic(rtw89_read32_mask, val, val == S_BE_BACAM_RST_DONE, + 1, 1000, false, + rtwdev, R_BE_RESPBA_CAM_CTRL, B_BE_BACAM_RST_MASK); + if (ret) + rtw89_err(rtwdev, "[ERR]bacam rst timeout\n"); + + return ret; +} + +#define PLD_RLS_MAX_PG 127 +#define RX_MAX_LEN_UNIT 512 +#define RX_SPEC_MAX_LEN (11454 + RX_MAX_LEN_UNIT) + +static int rmac_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 rx_min_qta, rx_max_len, rx_max_pg; + u16 val16; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + if (mac_idx == RTW89_MAC_0) { + ret = rst_bacam_be(rtwdev); + if (ret) + return ret; + } + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_DLK_PROTECT_CTL, mac_idx); + val16 = rtw89_read16(rtwdev, reg); + val16 = u16_replace_bits(val16, TRXCFG_RMAC_DATA_TO, B_BE_RX_DLK_DATA_TIME_MASK); + val16 = u16_replace_bits(val16, TRXCFG_RMAC_CCA_TO, B_BE_RX_DLK_CCA_TIME_MASK); + val16 |= B_BE_RX_DLK_RST_EN; + rtw89_write16(rtwdev, reg, val16); + + if (mac_idx == RTW89_MAC_0) + rx_min_qta = rtwdev->mac.dle_info.c0_rx_qta; + else + rx_min_qta = rtwdev->mac.dle_info.c1_rx_qta; + rx_max_pg = min_t(u32, rx_min_qta, PLD_RLS_MAX_PG); + rx_max_len = rx_max_pg * rtwdev->mac.dle_info.ple_pg_size; + rx_max_len = min_t(u32, rx_max_len, RX_SPEC_MAX_LEN); + rx_max_len /= RX_MAX_LEN_UNIT; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RX_FLTR_OPT, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_RX_MPDU_MAX_LEN_MASK, rx_max_len); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PLCP_HDR_FLTR, mac_idx); + rtw89_write8_clr(rtwdev, reg, B_BE_VHT_SU_SIGB_CRC_CHK); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RCR, mac_idx); + rtw89_write16_set(rtwdev, reg, B_BE_BUSY_CHKSN); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RX_PLCP_EXT_OPTION_1, mac_idx); + rtw89_write16_set(rtwdev, reg, B_BE_PLCP_SU_PSDU_LEN_SRC); + + return 0; +} + +static int resp_pktctl_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + struct rtw89_mac_dle_rsvd_qt_cfg qt_cfg; + enum rtw89_mac_dle_rsvd_qt_type type; + u32 reg; + int ret; + + if (mac_idx == RTW89_MAC_1) + type = DLE_RSVD_QT_B1_CSI; + else + type = DLE_RSVD_QT_B0_CSI; + + ret = rtw89_mac_get_dle_rsvd_qt_cfg(rtwdev, type, &qt_cfg); + if (ret) { + rtw89_err(rtwdev, "get dle rsvd qt %d cfg fail %d\n", type, ret); + return ret; + } + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RESP_CSI_RESERVED_PAGE, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_CSI_RESERVED_START_PAGE_MASK, qt_cfg.pktid); + rtw89_write32_mask(rtwdev, reg, B_BE_CSI_RESERVED_PAGE_NUM_MASK, qt_cfg.pg_num); + + return 0; +} + +static int cmac_com_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + if (mac_idx == RTW89_MAC_0) { + val32 = rtw89_read32(rtwdev, R_BE_TX_SUB_BAND_VALUE); + val32 = u32_replace_bits(val32, S_BE_TXSB_20M_8, B_BE_TXSB_20M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_40M_4, B_BE_TXSB_40M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_80M_2, B_BE_TXSB_80M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_160M_1, B_BE_TXSB_160M_MASK); + rtw89_write32(rtwdev, R_BE_TX_SUB_BAND_VALUE, val32); + } else { + val32 = rtw89_read32(rtwdev, R_BE_TX_SUB_BAND_VALUE_C1); + val32 = u32_replace_bits(val32, S_BE_TXSB_20M_2, B_BE_TXSB_20M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_40M_1, B_BE_TXSB_40M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_80M_0, B_BE_TXSB_80M_MASK); + val32 = u32_replace_bits(val32, S_BE_TXSB_160M_0, B_BE_TXSB_160M_MASK); + rtw89_write32(rtwdev, R_BE_TX_SUB_BAND_VALUE_C1, val32); + } + + return 0; +} + +static int ptcl_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + u8 val8; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + if (is_qta_poh(rtwdev)) { + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_SIFS_SETTING, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, S_AX_CTS2S_TH_1K, + B_BE_HW_CTS2SELF_PKT_LEN_TH_MASK); + val32 = u32_replace_bits(val32, S_AX_CTS2S_TH_SEC_256B, + B_BE_HW_CTS2SELF_PKT_LEN_TH_TWW_MASK); + val32 |= B_BE_HW_CTS2SELF_EN; + rtw89_write32(rtwdev, reg, val32); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_FSM_MON, mac_idx); + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, S_AX_PTCL_TO_2MS, + B_BE_PTCL_TX_ARB_TO_THR_MASK); + val32 &= ~B_BE_PTCL_TX_ARB_TO_MODE; + rtw89_write32(rtwdev, reg, val32); + } + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_COMMON_SETTING_0, mac_idx); + val8 = rtw89_read8(rtwdev, reg); + val8 |= B_BE_CMAC_TX_MODE_0 | B_BE_CMAC_TX_MODE_1; + val8 &= ~(B_BE_PTCL_TRIGGER_SS_EN_0 | + B_BE_PTCL_TRIGGER_SS_EN_1 | + B_BE_PTCL_TRIGGER_SS_EN_UL); + rtw89_write8(rtwdev, reg, val8); + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_AMPDU_AGG_LIMIT, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_BE_AMPDU_MAX_TIME_MASK, AMPDU_MAX_TIME); + + return 0; +} + +static int cmac_dma_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 val32; + u32 reg; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_RX_CTRL_1, mac_idx); + + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, WLCPU_RXCH2_QID, + B_BE_RXDMA_TXRPT_QUEUE_ID_SW_MASK); + val32 = u32_replace_bits(val32, WLCPU_RXCH2_QID, + B_BE_RXDMA_F2PCMDRPT_QUEUE_ID_SW_MASK); + rtw89_write32(rtwdev, reg, val32); + + return 0; +} + +static int cmac_init_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + int ret; + + ret = scheduler_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d SCH init %d\n", mac_idx, ret); + return ret; + } + + ret = addr_cam_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d ADDR_CAM reset %d\n", mac_idx, + ret); + return ret; + } + + ret = rx_fltr_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d RX filter init %d\n", mac_idx, + ret); + return ret; + } + + ret = cca_ctrl_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d CCA CTRL init %d\n", mac_idx, + ret); + return ret; + } + + ret = nav_ctrl_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d NAV CTRL init %d\n", mac_idx, + ret); + return ret; + } + + ret = spatial_reuse_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d Spatial Reuse init %d\n", + mac_idx, ret); + return ret; + } + + ret = tmac_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d TMAC init %d\n", mac_idx, ret); + return ret; + } + + ret = trxptcl_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d TRXPTCL init %d\n", mac_idx, ret); + return ret; + } + + ret = rmac_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d RMAC init %d\n", mac_idx, ret); + return ret; + } + + ret = resp_pktctl_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d resp pktctl init %d\n", mac_idx, ret); + return ret; + } + + ret = cmac_com_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d Com init %d\n", mac_idx, ret); + return ret; + } + + ret = ptcl_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d PTCL init %d\n", mac_idx, ret); + return ret; + } + + ret = cmac_dma_init_be(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d DMA init %d\n", mac_idx, ret); + return ret; + } + + return ret; +} + +static int tx_idle_poll_band_be(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + u32 reg; + u8 val8; + int ret; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(rtwdev, R_BE_PTCL_TX_CTN_SEL, mac_idx); + + ret = read_poll_timeout_atomic(rtw89_read8, val8, !(val8 & B_BE_PTCL_BUSY), + 30, 66000, false, rtwdev, reg); + + return ret; +} + static int dle_buf_req_be(struct rtw89_dev *rtwdev, u16 buf_len, bool wd, u16 *pkt_id) { u32 val, reg; @@ -650,6 +1449,275 @@ static int set_cpuio_be(struct rtw89_dev *rtwdev, return 0; } +static int preload_init_be(struct rtw89_dev *rtwdev, u8 mac_idx, + enum rtw89_qta_mode mode) +{ + u32 max_preld_size, min_rsvd_size; + u32 val32; + u32 reg; + + max_preld_size = mac_idx == RTW89_MAC_0 ? + PRELD_B0_ENT_NUM : PRELD_B1_ENT_NUM; + max_preld_size *= PRELD_AMSDU_SIZE; + + reg = mac_idx == RTW89_MAC_0 ? R_BE_TXPKTCTL_B0_PRELD_CFG0 : + R_BE_TXPKTCTL_B1_PRELD_CFG0; + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, max_preld_size, B_BE_B0_PRELD_USEMAXSZ_MASK); + val32 |= B_BE_B0_PRELD_FEN; + rtw89_write32(rtwdev, reg, val32); + + min_rsvd_size = PRELD_AMSDU_SIZE; + reg = mac_idx == RTW89_MAC_0 ? R_BE_TXPKTCTL_B0_PRELD_CFG1 : + R_BE_TXPKTCTL_B1_PRELD_CFG1; + val32 = rtw89_read32(rtwdev, reg); + val32 = u32_replace_bits(val32, PRELD_NEXT_WND, B_BE_B0_PRELD_NXT_TXENDWIN_MASK); + val32 = u32_replace_bits(val32, min_rsvd_size, B_BE_B0_PRELD_NXT_RSVMINSZ_MASK); + rtw89_write32(rtwdev, reg, val32); + + return 0; +} + +static int dbcc_bb_ctrl_be(struct rtw89_dev *rtwdev, bool bb1_en) +{ + return 0; +} + +static int enable_imr_be(struct rtw89_dev *rtwdev, u8 mac_idx, + enum rtw89_mac_hwmod_sel sel) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_imr_table *table; + const struct rtw89_reg_imr *reg; + u32 addr; + u32 val; + int i; + + if (sel == RTW89_DMAC_SEL) + table = chip->imr_dmac_table; + else if (sel == RTW89_CMAC_SEL) + table = chip->imr_cmac_table; + else + return -EINVAL; + + for (i = 0; i < table->n_regs; i++) { + reg = &table->regs[i]; + addr = rtw89_mac_reg_by_idx(rtwdev, reg->addr, mac_idx); + + val = rtw89_read32(rtwdev, addr); + val &= ~reg->clr; + val |= reg->set; + rtw89_write32(rtwdev, addr, val); + } + + return 0; +} + +static void err_imr_ctrl_be(struct rtw89_dev *rtwdev, bool en) +{ + u32 v32_dmac = en ? DMAC_ERR_IMR_EN : DMAC_ERR_IMR_DIS; + u32 v32_cmac0 = en ? CMAC0_ERR_IMR_EN : CMAC0_ERR_IMR_DIS; + u32 v32_cmac1 = en ? CMAC1_ERR_IMR_EN : CMAC1_ERR_IMR_DIS; + + v32_dmac &= ~B_BE_DMAC_NOTX_ERR_INT_EN; + + rtw89_write32(rtwdev, R_BE_DMAC_ERR_IMR, v32_dmac); + rtw89_write32(rtwdev, R_BE_CMAC_ERR_IMR, v32_cmac0); + + if (rtwdev->dbcc_en) + rtw89_write32(rtwdev, R_BE_CMAC_ERR_IMR_C1, v32_cmac1); +} + +static int band1_enable_be(struct rtw89_dev *rtwdev) +{ + int ret; + + ret = tx_idle_poll_band_be(rtwdev, RTW89_MAC_0); + if (ret) { + rtw89_err(rtwdev, "[ERR]tx idle poll %d\n", ret); + return ret; + } + + ret = rtw89_mac_dle_quota_change(rtwdev, rtwdev->mac.qta_mode); + if (ret) { + rtw89_err(rtwdev, "[ERR]DLE quota change %d\n", ret); + return ret; + } + + ret = preload_init_be(rtwdev, RTW89_MAC_1, rtwdev->mac.qta_mode); + if (ret) { + rtw89_err(rtwdev, "[ERR]preload init B1 %d\n", ret); + return ret; + } + + ret = cmac_func_en_be(rtwdev, RTW89_MAC_1, true); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d func en %d\n", RTW89_MAC_1, ret); + return ret; + } + + ret = cmac_init_be(rtwdev, RTW89_MAC_1); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d init %d\n", RTW89_MAC_1, ret); + return ret; + } + + ret = dbcc_bb_ctrl_be(rtwdev, true); + if (ret) { + rtw89_err(rtwdev, "[ERR]enable bb 1 %d\n", ret); + return ret; + } + + ret = enable_imr_be(rtwdev, RTW89_MAC_1, RTW89_CMAC_SEL); + if (ret) { + rtw89_err(rtwdev, "[ERR] enable CMAC1 IMR %d\n", ret); + return ret; + } + + return 0; +} + +static int band1_disable_be(struct rtw89_dev *rtwdev) +{ + int ret; + + ret = dbcc_bb_ctrl_be(rtwdev, false); + if (ret) { + rtw89_err(rtwdev, "[ERR]disable bb 1 %d\n", ret); + return ret; + } + + ret = cmac_func_en_be(rtwdev, RTW89_MAC_1, false); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d func dis %d\n", RTW89_MAC_1, ret); + return ret; + } + + ret = rtw89_mac_dle_quota_change(rtwdev, rtwdev->mac.qta_mode); + if (ret) { + rtw89_err(rtwdev, "[ERR]DLE quota change %d\n", ret); + return ret; + } + + return 0; +} + +static int dbcc_enable_be(struct rtw89_dev *rtwdev, bool enable) +{ + int ret; + + if (enable) { + ret = band1_enable_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR] band1_enable %d\n", ret); + return ret; + } + + if (test_bit(RTW89_FLAG_FW_RDY, rtwdev->flags)) { + ret = rtw89_fw_h2c_notify_dbcc(rtwdev, true); + if (ret) { + rtw89_err(rtwdev, "%s:[ERR]notfify dbcc1 fail %d\n", + __func__, ret); + return ret; + } + } + } else { + if (test_bit(RTW89_FLAG_FW_RDY, rtwdev->flags)) { + ret = rtw89_fw_h2c_notify_dbcc(rtwdev, false); + if (ret) { + rtw89_err(rtwdev, "%s:[ERR]notfify dbcc1 fail %d\n", + __func__, ret); + return ret; + } + } + + ret = band1_disable_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR] band1_disable %d\n", ret); + return ret; + } + } + + return 0; +} + +static int set_host_rpr_be(struct rtw89_dev *rtwdev) +{ + u32 val32; + u32 mode; + u32 fltr; + bool poh; + + poh = is_qta_poh(rtwdev); + + if (poh) { + mode = RTW89_RPR_MODE_POH; + fltr = S_BE_WDRLS_FLTR_TXOK | S_BE_WDRLS_FLTR_RTYLMT | + S_BE_WDRLS_FLTR_LIFTIM | S_BE_WDRLS_FLTR_MACID; + } else { + mode = RTW89_RPR_MODE_STF; + fltr = 0; + } + + rtw89_write32_mask(rtwdev, R_BE_WDRLS_CFG, B_BE_WDRLS_MODE_MASK, mode); + + val32 = rtw89_read32(rtwdev, R_BE_RLSRPT0_CFG1); + val32 = u32_replace_bits(val32, fltr, B_BE_RLSRPT0_FLTR_MAP_MASK); + val32 = u32_replace_bits(val32, 30, B_BE_RLSRPT0_AGGNUM_MASK); + val32 = u32_replace_bits(val32, 255, B_BE_RLSRPT0_TO_MASK); + rtw89_write32(rtwdev, R_BE_RLSRPT0_CFG1, val32); + + return 0; +} + +static int trx_init_be(struct rtw89_dev *rtwdev) +{ + enum rtw89_qta_mode qta_mode = rtwdev->mac.qta_mode; + int ret; + + ret = dmac_init_be(rtwdev, 0); + if (ret) { + rtw89_err(rtwdev, "[ERR]DMAC init %d\n", ret); + return ret; + } + + ret = cmac_init_be(rtwdev, 0); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d init %d\n", 0, ret); + return ret; + } + + if (rtw89_mac_is_qta_dbcc(rtwdev, qta_mode)) { + ret = dbcc_enable_be(rtwdev, true); + if (ret) { + rtw89_err(rtwdev, "[ERR]dbcc_enable init %d\n", ret); + return ret; + } + } + + ret = enable_imr_be(rtwdev, RTW89_MAC_0, RTW89_DMAC_SEL); + if (ret) { + rtw89_err(rtwdev, "[ERR] enable DMAC IMR %d\n", ret); + return ret; + } + + ret = enable_imr_be(rtwdev, RTW89_MAC_0, RTW89_CMAC_SEL); + if (ret) { + rtw89_err(rtwdev, "[ERR] to enable CMAC0 IMR %d\n", ret); + return ret; + } + + err_imr_ctrl_be(rtwdev, true); + + ret = set_host_rpr_be(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR] set host rpr %d\n", ret); + return ret; + } + + return 0; +} + static bool rtw89_mac_get_txpwr_cr_be(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx, u32 reg_base, u32 *cr) @@ -1162,12 +2230,16 @@ const struct rtw89_mac_gen_def rtw89_mac_gen_be = { }, .check_mac_en = rtw89_mac_check_mac_en_be, + .sys_init = sys_init_be, + .trx_init = trx_init_be, .hci_func_en = rtw89_mac_hci_func_en_be, .dmac_func_pre_en = rtw89_mac_dmac_func_pre_en_be, .dle_func_en = dle_func_en_be, .dle_clk_en = dle_clk_en_be, .bf_assoc = rtw89_mac_bf_assoc_be, + .typ_fltr_opt = rtw89_mac_typ_fltr_opt_be, + .dle_mix_cfg = dle_mix_cfg_be, .chk_dle_rdy = chk_dle_rdy_be, .dle_buf_req = dle_buf_req_be, diff --git a/drivers/net/wireless/realtek/rtw89/reg.h b/drivers/net/wireless/realtek/rtw89/reg.h index 65d8a0f36700..8456e2b0c14f 100644 --- a/drivers/net/wireless/realtek/rtw89/reg.h +++ b/drivers/net/wireless/realtek/rtw89/reg.h @@ -4924,6 +4924,24 @@ B_BE_CR_WRFF_OVERFLOW_ERR_INT_EN | \ B_BE_CR_WRFF_UNDERFLOW_ERR_INT_EN) +#define R_BE_DISP_FWD_WLAN_0 0x8938 +#define B_BE_FWD_WLAN_CPU_TYPE_13_MASK GENMASK(31, 30) +#define B_BE_FWD_WLAN_CPU_TYPE_12_MASK GENMASK(29, 28) +#define B_BE_FWD_WLAN_CPU_TYPE_11_MASK GENMASK(27, 26) +#define B_BE_FWD_WLAN_CPU_TYPE_10_MASK GENMASK(25, 24) +#define B_BE_FWD_WLAN_CPU_TYPE_9_MASK GENMASK(23, 22) +#define B_BE_FWD_WLAN_CPU_TYPE_8_MASK GENMASK(21, 20) +#define B_BE_FWD_WLAN_CPU_TYPE_7_MASK GENMASK(19, 18) +#define B_BE_FWD_WLAN_CPU_TYPE_6_MASK GENMASK(17, 16) +#define B_BE_FWD_WLAN_CPU_TYPE_5_MASK GENMASK(15, 14) +#define B_BE_FWD_WLAN_CPU_TYPE_4_MASK GENMASK(13, 12) +#define B_BE_FWD_WLAN_CPU_TYPE_3_MASK GENMASK(11, 10) +#define B_BE_FWD_WLAN_CPU_TYPE_2_MASK GENMASK(9, 8) +#define B_BE_FWD_WLAN_CPU_TYPE_1_MASK GENMASK(7, 6) +#define B_BE_FWD_WLAN_CPU_TYPE_0_CTL_MASK GENMASK(5, 4) +#define B_BE_FWD_WLAN_CPU_TYPE_0_MNG_MASK GENMASK(3, 2) +#define B_BE_FWD_WLAN_CPU_TYPE_0_DATA_MASK GENMASK(1, 0) + #define R_BE_WDE_PKTBUF_CFG 0x8C08 #define B_BE_WDE_FREE_PAGE_NUM_MASK GENMASK(28, 16) #define B_BE_WDE_START_BOUND_MASK GENMASK(14, 8) @@ -5387,11 +5405,47 @@ #define B_BE_PKTIN_ERR_IMR_SET (B_BE_SW_MERGE_ERR_INT_EN | \ B_BE_GET_NULL_PKTID_ERR_INT_EN) +#define R_BE_HDR_SHCUT_SETTING 0x9B00 +#define B_BE_TX_ADDR_MLD_TO_LIK BIT(4) +#define B_BE_TX_HW_SEC_HDR_EN BIT(3) +#define B_BE_TX_MAC_MPDU_PROC_EN BIT(2) +#define B_BE_TX_HW_ACK_POLICY_EN BIT(1) +#define B_BE_TX_HW_SEQ_EN BIT(0) + #define R_BE_MPDU_TX_ERR_IMR 0x9BF4 #define B_BE_TX_TIMEOUT_ERR_EN BIT(0) #define B_BE_MPDU_TX_ERR_IMR_CLR B_BE_TX_TIMEOUT_ERR_EN #define B_BE_MPDU_TX_ERR_IMR_SET 0 +#define R_BE_MPDU_PROC 0x9C00 +#define B_BE_PORT_SEL BIT(29) +#define B_BE_WPKT_WLANCPU_QSEL_MASK GENMASK(28, 27) +#define B_BE_WPKT_DATACPU_QSEL_MASK GENMASK(26, 25) +#define B_BE_WPKT_FW_RLS BIT(24) +#define B_BE_FWD_RPKT_MASK GENMASK(23, 16) +#define B_BE_FWD_WPKT_MASK GENMASK(15, 8) +#define B_BE_RXFWD_PRIO_MASK GENMASK(5, 4) +#define B_BE_RXFWD_EN BIT(3) +#define B_BE_DROP_NONDMA_PPDU BIT(2) +#define B_BE_APPEND_FCS BIT(0) + +#define R_BE_CUT_AMSDU_CTRL 0x9C94 +#define B_BE_EN_CUT_AMSDU BIT(31) +#define B_BE_CUT_AMSDU_CHKLEN_EN BIT(30) +#define B_BE_CA_CHK_ADDRCAM_EN BIT(29) +#define B_BE_MPDU_CUT_CTRL_EN BIT(24) +#define B_BE_CUT_AMSDU_CHKLEN_L_TH_MASK GENMASK(23, 16) +#define B_BE_CUT_AMSDU_CHKLEN_H_TH_MASK GENMASK(15, 0) + +#define R_BE_RX_HDRTRNS 0x9CC0 +#define B_BE_RX_MGN_MLD_ADDR_EN BIT(6) +#define B_BE_HDR_INFO_MASK GENMASK(5, 4) +#define B_BE_HC_ADDR_HIT_EN BIT(3) +#define B_BE_RX_ADDR_LINK_TO_MLO BIT(2) +#define B_BE_HDR_CNV BIT(1) +#define B_BE_RX_HDR_CNV_EN BIT(0) +#define TRXCFG_MPDU_PROC_RX_HDR_CONV 0x00000000 + #define R_BE_MPDU_RX_ERR_IMR 0x9CF4 #define B_BE_LEN_ERR_IMR BIT(3) #define B_BE_TIMEOUT_ERR_IMR BIT(1) @@ -5472,6 +5526,22 @@ #define B_BE_RX_HANG_ERROR BIT(1) #define B_BE_TX_HANG_ERROR BIT(0) +#define R_BE_TXPKTCTL_MPDUINFO_CFG 0x9F10 +#define B_BE_MPDUINFO_FEN BIT(31) +#define B_BE_MPDUINFO_PKTID_MASK GENMASK(27, 16) +#define B_BE_MPDUINFO_B1_BADDR_MASK GENMASK(5, 0) +#define MPDU_INFO_B1_OFST 18 + +#define R_BE_TXPKTCTL_B0_PRELD_CFG0 0x9F48 +#define B_BE_B0_PRELD_FEN BIT(31) +#define B_BE_B0_PRELD_USEMAXSZ_MASK GENMASK(25, 16) +#define B_BE_B0_PRELD_CAM_G1ENTNUM_MASK GENMASK(12, 8) +#define B_BE_B0_PRELD_CAM_G0ENTNUM_MASK GENMASK(4, 0) + +#define R_BE_TXPKTCTL_B0_PRELD_CFG1 0x9F4C +#define B_BE_B0_PRELD_NXT_TXENDWIN_MASK GENMASK(11, 8) +#define B_BE_B0_PRELD_NXT_RSVMINSZ_MASK GENMASK(7, 0) + #define R_BE_TXPKTCTL_B0_ERRFLAG_IMR 0x9F78 #define B_BE_B0_IMR_DBG_USRCTL_RLSBMPLEN BIT(25) #define B_BE_B0_IMR_DBG_USRCTL_RDNRLSCMD BIT(24) @@ -5502,6 +5572,16 @@ B_BE_B0_IMR_ERR_PRELD_RLSPKTSZERR | \ B_BE_B0_IMR_ERR_PRELD_ENTNUMCFG) +#define R_BE_TXPKTCTL_B1_PRELD_CFG0 0x9F88 +#define B_BE_B1_PRELD_FEN BIT(31) +#define B_BE_B1_PRELD_USEMAXSZ_MASK GENMASK(25, 16) +#define B_BE_B1_PRELD_CAM_G1ENTNUM_MASK GENMASK(12, 8) +#define B_BE_B1_PRELD_CAM_G0ENTNUM_MASK GENMASK(4, 0) + +#define R_BE_TXPKTCTL_B1_PRELD_CFG1 0x9F8C +#define B_BE_B1_PRELD_NXT_TXENDWIN_MASK GENMASK(11, 8) +#define B_BE_B1_PRELD_NXT_RSVMINSZ_MASK GENMASK(7, 0) + #define R_BE_TXPKTCTL_B1_ERRFLAG_IMR 0x9FB8 #define B_BE_B1_IMR_DBG_USRCTL_RLSBMPLEN BIT(25) #define B_BE_B1_IMR_DBG_USRCTL_RDNRLSCMD BIT(24) @@ -5532,6 +5612,12 @@ B_BE_B1_IMR_ERR_PRELD_RLSPKTSZERR | \ B_BE_B1_IMR_ERR_PRELD_ENTNUMCFG) +#define R_BE_MLO_INIT_CTL 0xA114 +#define B_BE_MLO_TABLE_INIT_DONE BIT(31) +#define B_BE_MLO_TABLE_CLR_DONE BIT(30) +#define B_BE_MLO_TABLE_REINIT BIT(23) +#define B_BE_MLO_TABLE_HW_FLAG_CLR BIT(22) + #define R_BE_MLO_ERR_IDCT_IMR 0xA128 #define B_BE_MLO_ERR_IDCT_IMR_0 BIT(31) #define B_BE_MLO_ERR_IDCT_IMR_1 BIT(30) @@ -5561,6 +5647,30 @@ #define B_BE_PLRLS_CTL_EVT01_ISR BIT(1) #define B_BE_PLRLS_CTL_FRZTO_ISR BIT(0) +#define R_BE_SS_CTRL 0xA310 +#define B_BE_SS_INIT_DONE BIT(31) +#define B_BE_WDE_STA_DIS BIT(30) +#define B_BE_WARM_INIT BIT(29) +#define B_BE_BAND_TRIG_EN BIT(28) +#define B_BE_RMAC_REQ_DIS BIT(27) +#define B_BE_DLYTX_SEL_MASK GENMASK(25, 24) +#define B_BE_WMM3_SWITCH_MASK GENMASK(23, 22) +#define B_BE_WMM2_SWITCH_MASK GENMASK(21, 20) +#define B_BE_WMM1_SWITCH_MASK GENMASK(19, 18) +#define B_BE_WMM0_SWITCH_MASK GENMASK(17, 16) +#define B_BE_STA_OPTION_CR BIT(15) +#define B_BE_EMLSR_STA_EMPTY_EN BIT(11) +#define B_BE_MLO_HW_CHGLINK_EN BIT(10) +#define B_BE_BAND1_TRIG_EN BIT(9) +#define B_BE_RMAC1_REQ_DIS BIT(8) +#define B_BE_MRT_SRAM_EN BIT(7) +#define B_BE_MRT_INIT_EN BIT(6) +#define B_BE_AVG_LENG_EN BIT(5) +#define B_BE_AVG_INIT_EN BIT(4) +#define B_BE_LENG_INIT_EN BIT(2) +#define B_BE_PMPA_INIT_EN BIT(1) +#define B_BE_SS_EN BIT(0) + #define R_BE_INTERRUPT_MASK_REG 0xA3F0 #define B_BE_PLE_B_PKTID_ERR_IMR BIT(2) #define B_BE_RPT_TIMEOUT_IMR BIT(1) @@ -5705,6 +5815,13 @@ #define B_BE_ADDRSRCH_EN BIT(1) #define B_BE_BTCOEX_EN BIT(0) +#define R_BE_CMAC_SHARE_ACQCHK_CFG_0 0x0E010 +#define B_BE_ACQCHK_ERR_FLAG_MASK GENMASK(31, 24) +#define B_BE_R_ACQCHK_ENTRY_IDX_SEL_MASK GENMASK(7, 4) +#define B_BE_MACID_ACQ_GRP1_CLR_P BIT(3) +#define B_BE_MACID_ACQ_GRP0_CLR_P BIT(2) +#define B_BE_R_MACID_ACQ_CHK_EN BIT(0) + #define R_BE_CMAC_FUNC_EN 0x10000 #define R_BE_CMAC_FUNC_EN_C1 0x14000 #define B_BE_CMAC_CRPRT BIT(31) @@ -5756,6 +5873,40 @@ B_BE_RMAC_CKEN | B_BE_TXTIME_CKEN | B_BE_RESP_PKTCTL_CKEN | \ B_BE_SIGB_CKEN) +#define R_BE_TX_SUB_BAND_VALUE 0x10088 +#define R_BE_TX_SUB_BAND_VALUE_C1 0x14088 +#define B_BE_PRI20_BITMAP_MASK GENMASK(31, 16) +#define BE_PRI20_BITMAP_MAX 15 +#define B_BE_TXSB_160M_MASK GENMASK(15, 12) +#define S_BE_TXSB_160M_0 0 +#define S_BE_TXSB_160M_1 1 +#define B_BE_TXSB_80M_MASK GENMASK(11, 8) +#define S_BE_TXSB_80M_0 0 +#define S_BE_TXSB_80M_2 2 +#define S_BE_TXSB_80M_4 4 +#define B_BE_TXSB_40M_MASK GENMASK(7, 4) +#define S_BE_TXSB_40M_0 0 +#define S_BE_TXSB_40M_1 1 +#define S_BE_TXSB_40M_4 4 +#define B_BE_TXSB_20M_MASK GENMASK(3, 0) +#define S_BE_TXSB_20M_8 8 +#define S_BE_TXSB_20M_4 4 +#define S_BE_TXSB_20M_2 2 + +#define R_BE_PTCL_RRSR0 0x1008C +#define R_BE_PTCL_RRSR0_C1 0x1408C +#define B_BE_RRSR_HE_MASK GENMASK(31, 24) +#define B_BE_RRSR_VHT_MASK GENMASK(23, 16) +#define B_BE_RRSR_HT_MASK GENMASK(15, 8) +#define B_BE_RRSR_OFDM_MASK GENMASK(7, 0) + +#define R_BE_PTCL_RRSR1 0x10090 +#define R_BE_PTCL_RRSR1_C1 0x14090 +#define B_BE_RRSR_EHT_MASK GENMASK(23, 16) +#define B_BE_RRSR_RATE_EN_MASK GENMASK(12, 8) +#define B_BE_RSC_MASK GENMASK(7, 6) +#define B_BE_RRSR_CCK_MASK GENMASK(3, 0) + #define R_BE_CMAC_ERR_IMR 0x10160 #define R_BE_CMAC_ERR_IMR_C1 0x14160 #define B_BE_CMAC_FW_ERR_IDCT_EN BIT(16) @@ -5843,6 +5994,55 @@ #define B_BE_P0_SYNC_PORT_SRC_SEL_MASK GENMASK(26, 24) #define B_BE_P0_TSFTR_SYNC_OFFSET_MASK GENMASK(18, 0) +#define R_BE_EDCA_BCNQ_PARAM 0x10324 +#define R_BE_EDCA_BCNQ_PARAM_C1 0x14324 +#define B_BE_BCNQ_CW_MASK GENMASK(31, 24) +#define B_BE_BCNQ_AIFS_MASK GENMASK(23, 16) +#define BCN_IFS_25US 0x19 +#define B_BE_PIFS_MASK GENMASK(15, 8) +#define B_BE_FORCE_BCN_IFS_MASK GENMASK(7, 0) + +#define R_BE_PREBKF_CFG_0 0x10338 +#define R_BE_PREBKF_CFG_0_C1 0x14338 +#define B_BE_100NS_TIME_MASK GENMASK(28, 24) +#define B_BE_RX_AIR_END_TIME_MASK GENMASK(22, 16) +#define B_BE_MACTX_LATENCY_MASK GENMASK(10, 8) +#define B_BE_PREBKF_TIME_MASK GENMASK(4, 0) + +#define R_BE_CCA_CFG_0 0x10340 +#define R_BE_CCA_CFG_0_C1 0x14340 +#define B_BE_R_SIFS_AGGR_TIME_V1_MASK GENMASK(31, 24) +#define B_BE_EDCCA_SEC160_EN BIT(23) +#define B_BE_EDCCA_SEC80_EN BIT(22) +#define B_BE_EDCCA_SEC40_EN BIT(21) +#define B_BE_EDCCA_SEC20_EN BIT(20) +#define B_BE_SEC160_EN BIT(19) +#define B_BE_CCA_BITMAP_EN BIT(18) +#define B_BE_TXPKTCTL_RST_EDCA_EN BIT(17) +#define B_BE_WMAC_RST_EDCA_EN BIT(16) +#define B_BE_TXFAIL_BRK_TXOP_EN BIT(11) +#define B_BE_EDCCA_PER20_BITMAP_SIFS_EN BIT(10) +#define B_BE_NO_GNT_WL_BRK_TXOP_EN BIT(9) +#define B_BE_NAV_BRK_TXOP_EN BIT(8) +#define B_BE_TX_NAV_EN BIT(7) +#define B_BE_BCN_IGNORE_EDCCA BIT(6) +#define B_BE_NO_GNT_WL_EN BIT(5) +#define B_BE_EDCCA_EN BIT(4) +#define B_BE_SEC80_EN BIT(3) +#define B_BE_SEC40_EN BIT(2) +#define B_BE_SEC20_EN BIT(1) +#define B_BE_CCA_EN BIT(0) + +#define R_BE_CTN_CFG_0 0x1034C +#define R_BE_CTN_CFG_0_C1 0x1434C +#define B_BE_OTHER_LINK_BKF_BLK_TX_THD_MASK GENMASK(30, 24) +#define B_BE_CCK_SIFS_COMP_MASK GENMASK(22, 16) +#define B_BE_PIFS_TIMEUNIT_MASK GENMASK(15, 14) +#define B_BE_PREBKF_TIME_NONAC_MASK GENMASK(12, 8) +#define B_BE_SR_TX_EN BIT(2) +#define B_BE_NAV_BLK_MGQ BIT(1) +#define B_BE_NAV_BLK_HGQ BIT(0) + #define R_BE_MUEDCA_BE_PARAM_0 0x10350 #define R_BE_MUEDCA_BK_PARAM_0 0x10354 #define R_BE_MUEDCA_VI_PARAM_0 0x10358 @@ -5855,6 +6055,63 @@ #define B_BE_SET_MUEDCATIMER_TF_0 BIT(4) #define B_BE_MUEDCA_EN_0 BIT(0) +#define R_BE_TB_CHK_CCA_NAV 0x103AC +#define R_BE_TB_CHK_CCA_NAV_C1 0x143AC +#define B_BE_TB_CHK_TX_NAV BIT(15) +#define B_BE_TB_CHK_INTRA_NAV BIT(14) +#define B_BE_TB_CHK_BASIC_NAV BIT(13) +#define B_BE_TB_CHK_NO_GNT_WL BIT(12) +#define B_BE_TB_CHK_EDCCA_S160 BIT(11) +#define B_BE_TB_CHK_EDCCA_S80 BIT(10) +#define B_BE_TB_CHK_EDCCA_S40 BIT(9) +#define B_BE_TB_CHK_EDCCA_S20 BIT(8) +#define B_BE_TB_CHK_CCA_S160 BIT(7) +#define B_BE_TB_CHK_CCA_S80 BIT(6) +#define B_BE_TB_CHK_CCA_S40 BIT(5) +#define B_BE_TB_CHK_CCA_S20 BIT(4) +#define B_BE_TB_CHK_EDCCA_BITMAP BIT(3) +#define B_BE_TB_CHK_CCA_BITMAP BIT(2) +#define B_BE_TB_CHK_EDCCA_P20 BIT(1) +#define B_BE_TB_CHK_CCA_P20 BIT(0) + +#define R_BE_HE_SIFS_CHK_CCA_NAV 0x103B4 +#define R_BE_HE_SIFS_CHK_CCA_NAV_C1 0x143B4 +#define B_BE_HE_SIFS_CHK_TX_NAV BIT(15) +#define B_BE_HE_SIFS_CHK_INTRA_NAV BIT(14) +#define B_BE_HE_SIFS_CHK_BASIC_NAV BIT(13) +#define B_BE_HE_SIFS_CHK_NO_GNT_WL BIT(12) +#define B_BE_HE_SIFS_CHK_EDCCA_S160 BIT(11) +#define B_BE_HE_SIFS_CHK_EDCCA_S80 BIT(10) +#define B_BE_HE_SIFS_CHK_EDCCA_S40 BIT(9) +#define B_BE_HE_SIFS_CHK_EDCCA_S20 BIT(8) +#define B_BE_HE_SIFS_CHK_CCA_S160 BIT(7) +#define B_BE_HE_SIFS_CHK_CCA_S80 BIT(6) +#define B_BE_HE_SIFS_CHK_CCA_S40 BIT(5) +#define B_BE_HE_SIFS_CHK_CCA_S20 BIT(4) +#define B_BE_HE_SIFS_CHK_EDCCA_BITMAP BIT(3) +#define B_BE_HE_SIFS_CHK_CCA_BITMAP BIT(2) +#define B_BE_HE_SIFS_CHK_EDCCA_P20 BIT(1) +#define B_BE_HE_SIFS_CHK_CCA_P20 BIT(0) + +#define R_BE_HE_CTN_CHK_CCA_NAV 0x103C4 +#define R_BE_HE_CTN_CHK_CCA_NAV_C1 0x143C4 +#define B_BE_HE_CTN_CHK_TX_NAV BIT(15) +#define B_BE_HE_CTN_CHK_INTRA_NAV BIT(14) +#define B_BE_HE_CTN_CHK_BASIC_NAV BIT(13) +#define B_BE_HE_CTN_CHK_NO_GNT_WL BIT(12) +#define B_BE_HE_CTN_CHK_EDCCA_S160 BIT(11) +#define B_BE_HE_CTN_CHK_EDCCA_S80 BIT(10) +#define B_BE_HE_CTN_CHK_EDCCA_S40 BIT(9) +#define B_BE_HE_CTN_CHK_EDCCA_S20 BIT(8) +#define B_BE_HE_CTN_CHK_CCA_S160 BIT(7) +#define B_BE_HE_CTN_CHK_CCA_S80 BIT(6) +#define B_BE_HE_CTN_CHK_CCA_S40 BIT(5) +#define B_BE_HE_CTN_CHK_CCA_S20 BIT(4) +#define B_BE_HE_CTN_CHK_EDCCA_BITMAP BIT(3) +#define B_BE_HE_CTN_CHK_CCA_BITMAP BIT(2) +#define B_BE_HE_CTN_CHK_EDCCA_P20 BIT(1) +#define B_BE_HE_CTN_CHK_CCA_P20 BIT(0) + #define R_BE_SCHEDULE_ERR_IMR 0x103E8 #define R_BE_SCHEDULE_ERR_IMR_C1 0x143E8 #define B_BE_FSM_TIMEOUT_ERR_INT_EN BIT(0) @@ -5980,12 +6237,51 @@ #define R_BE_PORT_HGQ_WINDOW_CFG 0x105A0 #define R_BE_PORT_HGQ_WINDOW_CFG_C1 0x145A0 +#define R_BE_PTCL_COMMON_SETTING_0 0x10800 +#define R_BE_PTCL_COMMON_SETTING_0_C1 0x14800 +#define B_BE_PCIE_MODE_MASK GENMASK(15, 14) +#define B_BE_CPUMGQ_LIFETIME_EN BIT(8) +#define B_BE_MGQ_LIFETIME_EN BIT(7) +#define B_BE_LIFETIME_EN BIT(6) +#define B_BE_DIS_PTCL_CLK_GATING BIT(5) +#define B_BE_PTCL_TRIGGER_SS_EN_UL BIT(4) +#define B_BE_PTCL_TRIGGER_SS_EN_1 BIT(3) +#define B_BE_PTCL_TRIGGER_SS_EN_0 BIT(2) +#define B_BE_CMAC_TX_MODE_1 BIT(1) +#define B_BE_CMAC_TX_MODE_0 BIT(0) + +#define R_BE_TB_PPDU_CTRL 0x1080C +#define R_BE_TB_PPDU_CTRL_C1 0x1480C +#define B_BE_TB_PPDU_BK_DIS BIT(15) +#define B_BE_TB_PPDU_BE_DIS BIT(14) +#define B_BE_TB_PPDU_VI_DIS BIT(13) +#define B_BE_TB_PPDU_VO_DIS BIT(12) +#define B_BE_QOSNULL_UPD_MUEDCA_EN BIT(3) +#define B_BE_TB_BYPASS_TXPWR BIT(2) +#define B_BE_SW_PREFER_AC_MASK GENMASK(1, 0) + +#define R_BE_AMPDU_AGG_LIMIT 0x10810 +#define R_BE_AMPDU_AGG_LIMIT_C1 0x14810 +#define B_BE_AMPDU_MAX_TIME_MASK GENMASK(31, 24) +#define AMPDU_MAX_TIME 0x9E +#define B_BE_RA_TRY_RATE_AGG_LMT_MASK GENMASK(23, 16) +#define B_BE_RTS_MAX_AGG_NUM_MASK GENMASK(15, 8) +#define B_BE_MAX_AGG_NUM_MASK GENMASK(7, 0) + #define R_BE_AGG_LEN_HT_0 0x10814 #define R_BE_AGG_LEN_HT_0_C1 0x14814 #define B_BE_AMPDU_MAX_LEN_HT_MASK GENMASK(31, 16) #define B_BE_RTS_TXTIME_TH_MASK GENMASK(15, 8) #define B_BE_RTS_LEN_TH_MASK GENMASK(7, 0) +#define R_BE_SIFS_SETTING 0x10824 +#define R_BE_SIFS_SETTING_C1 0x14824 +#define B_BE_HW_CTS2SELF_PKT_LEN_TH_MASK GENMASK(31, 24) +#define B_BE_HW_CTS2SELF_PKT_LEN_TH_TWW_MASK GENMASK(23, 18) +#define B_BE_HW_CTS2SELF_EN BIT(16) +#define B_BE_SPEC_SIFS_OFDM_PTCL_MASK GENMASK(15, 8) +#define B_BE_SPEC_SIFS_CCK_PTCL_MASK GENMASK(7, 0) + #define R_BE_MBSSID_DROP_0 0x1083C #define R_BE_MBSSID_DROP_0_C1 0x1483C #define B_BE_GI_LTF_FB_SEL BIT(30) @@ -6084,6 +6380,24 @@ #define B_BE_TXPRT_FULL_DROP_ERR BIT(9) #define B_BE_F2PCMDRPT_FULL_DROP_ERR BIT(8) +#define R_BE_PTCL_FSM_MON 0x108E8 +#define R_BE_PTCL_FSM_MON_C1 0x148E8 +#define B_BE_PTCL_FSM2_TO_MODE BIT(30) +#define B_BE_PTCL_FSM2_TO_THR_MASK GENMASK(29, 24) +#define B_BE_PTCL_FSM1_TO_MODE BIT(22) +#define B_BE_PTCL_FSM1_TO_THR_MASK GENMASK(21, 16) +#define B_BE_PTCL_FSM0_TO_MODE BIT(14) +#define B_BE_PTCL_FSM0_TO_THR_MASK GENMASK(13, 8) +#define B_BE_PTCL_TX_ARB_TO_MODE BIT(6) +#define B_BE_PTCL_TX_ARB_TO_THR_MASK GENMASK(5, 0) + +#define R_BE_PTCL_TX_CTN_SEL 0x108EC +#define R_BE_PTCL_TX_CTN_SEL_C1 0x148EC +#define B_BE_PTCL_TXOP_STAT BIT(8) +#define B_BE_PTCL_BUSY BIT(7) +#define B_BE_PTCL_DROP BIT(5) +#define B_BE_PTCL_TX_QUEUE_IDX_MASK GENMASK(4, 0) + #define R_BE_RX_ERROR_FLAG 0x10C00 #define R_BE_RX_ERROR_FLAG_C1 0x14C00 #define B_BE_RX_CSI_NOT_RELEASE_ERROR BIT(31) @@ -6198,6 +6512,15 @@ B_BE_RX_RU0_FSM_HANG_ERROR_IMR | \ B_BE_RX_GET_NULL_PKT_ERROR_IMR) +#define R_BE_RX_CTRL_1 0x10C0C +#define R_BE_RX_CTRL_1_C1 0x14C0C +#define B_BE_RXDMA_TXRPT_QUEUE_ID_SW_MASK GENMASK(30, 25) +#define B_BE_RXDMA_F2PCMDRPT_QUEUE_ID_SW_MASK GENMASK(23, 18) +#define B_BE_RXDMA_TXRPT_PORT_ID_SW_MASK GENMASK(17, 14) +#define B_BE_RXDMA_F2PCMDRPT_PORT_ID_SW_MASK GENMASK(13, 10) +#define B_BE_DBG_SEL_MASK GENMASK(1, 0) +#define WLCPU_RXCH2_QID 0xA + #define R_BE_TX_ERROR_FLAG 0x10C6C #define R_BE_TX_ERROR_FLAG_C1 0x14C6C #define B_BE_TX_RU0_FSM_HANG_ERROR BIT(31) @@ -6353,6 +6676,15 @@ #define B_BE_UPD_HGQMD BIT(1) #define B_BE_UPD_TIMIE BIT(0) +#define R_BE_WMTX_TCR_BE_4 0x10E2C +#define R_BE_WMTX_TCR_BE_4_C1 0x14E2C +#define B_BE_UL_EHT_MUMIMO_LTF_MODE BIT(30) +#define B_BE_UL_HE_MUMIMO_LTF_MODE BIT(29) +#define B_BE_EHT_HE_PPDU_4XLTF_ZLD_USTIMER_MASK GENMASK(28, 24) +#define B_BE_EHT_HE_PPDU_2XLTF_ZLD_USTIMER_MASK GENMASK(20, 16) +#define B_BE_NON_LEGACY_PPDU_ZLD_USTIMER_MASK GENMASK(12, 8) +#define B_BE_LEGACY_PPDU_ZLD_USTIMER_MASK GENMASK(4, 0) + #define R_BE_RSP_CHK_SIG 0x11000 #define R_BE_RSP_CHK_SIG_C1 0x15000 #define B_BE_RSP_STATIC_RTS_CHK_SERV_BW_EN BIT(30) @@ -6385,6 +6717,46 @@ #define WMAC_SPEC_SIFS_OFDM_1115E 0x11 #define B_BE_WMAC_SPEC_SIFS_CCK_MASK GENMASK(7, 0) +#define R_BE_TRXPTCL_RESP_1 0x11008 +#define R_BE_TRXPTCL_RESP_1_C1 0x15008 +#define B_BE_WMAC_RESP_SR_MODE_EN BIT(31) +#define B_BE_FTM_RRSR_RATE_EN_MASK GENMASK(28, 24) +#define B_BE_NESS_MASK GENMASK(23, 22) +#define B_BE_WMAC_RESP_DOPPLEB_BE_EN BIT(21) +#define B_BE_WMAC_RESP_DCM_EN BIT(20) +#define B_BE_WMAC_CLR_ABORT_RESP_TX_CNT BIT(15) +#define B_BE_WMAC_RESP_REF_RATE_SEL BIT(12) +#define B_BE_WMAC_RESP_REF_RATE_MASK GENMASK(11, 0) + +#define R_BE_MAC_LOOPBACK 0x11020 +#define R_BE_MAC_LOOPBACK_C1 0x15020 +#define B_BE_MACLBK_DIS_GCLK BIT(30) +#define B_BE_MACLBK_STS_EN BIT(29) +#define B_BE_MACLBK_RDY_PERIOD_MASK GENMASK(28, 17) +#define B_BE_MACLBK_PLCP_DLY_MASK GENMASK(16, 8) +#define S_BE_MACLBK_PLCP_DLY_DEF 0x28 +#define B_BE_MACLBK_RDY_NUM_MASK GENMASK(7, 3) +#define B_BE_MACLBK_EN BIT(0) + +#define R_BE_WMAC_NAV_CTL 0x11080 +#define R_BE_WMAC_NAV_CTL_C1 0x15080 +#define B_BE_WMAC_NAV_UPPER_EN BIT(26) +#define B_BE_WMAC_0P125US_TIMER_MASK GENMASK(25, 18) +#define B_BE_WMAC_PLCP_UP_NAV_EN BIT(17) +#define B_BE_WMAC_TF_UP_NAV_EN BIT(16) +#define B_BE_WMAC_NAV_UPPER_MASK GENMASK(15, 8) +#define NAV_25MS 0xC4 +#define B_BE_WMAC_RTS_RST_DUR_MASK GENMASK(7, 0) + +#define R_BE_RXTRIG_TEST_USER_2 0x110B0 +#define R_BE_RXTRIG_TEST_USER_2_C1 0x150B0 +#define B_BE_RXTRIG_MACID_MASK GENMASK(31, 24) +#define B_BE_RXTRIG_RU26_DIS BIT(21) +#define B_BE_RXTRIG_FCSCHK_EN BIT(20) +#define B_BE_RXTRIG_PORT_SEL_MASK GENMASK(19, 17) +#define B_BE_RXTRIG_EN BIT(16) +#define B_BE_RXTRIG_USERINFO_2_MASK GENMASK(15, 0) + #define R_BE_TRXPTCL_ERROR_INDICA_MASK 0x110BC #define R_BE_TRXPTCL_ERROR_INDICA_MASK_C1 0x150BC #define B_BE_WMAC_FTM_TIMEOUT_MODE BIT(30) @@ -6526,6 +6898,103 @@ #define B_BE_BFMEE_HT_CSI_RATE_MASK GENMASK(7, 0) #define CSI_INIT_RATE_EHT 0x3 +#define R_BE_WMAC_ACK_BA_RESP_LEGACY 0x11200 +#define R_BE_WMAC_ACK_BA_RESP_LEGACY_C1 0x15200 +#define B_BE_ACK_BA_RESP_LEGACY_CHK_NSTR BIT(16) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_TX_NAV BIT(15) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_INTRA_NAV BIT(14) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_BASIC_NAV BIT(13) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_BTCCA BIT(12) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_EDCCA160 BIT(11) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_EDCCA80 BIT(10) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_EDCCA40 BIT(9) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_EDCCA20 BIT(8) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_EDCCA_PER20_BMP BIT(7) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_CCA_PER20_BMP BIT(6) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_CCA160 BIT(5) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_CCA80 BIT(4) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_CCA40 BIT(3) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_SEC_CCA20 BIT(2) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_EDCCA BIT(1) +#define B_BE_ACK_BA_RESP_LEGACY_CHK_CCA BIT(0) + +#define R_BE_WMAC_ACK_BA_RESP_HE 0x11204 +#define R_BE_WMAC_ACK_BA_RESP_HE_C1 0x15204 +#define B_BE_ACK_BA_RESP_HE_CHK_NSTR BIT(16) +#define B_BE_ACK_BA_RESP_HE_CHK_TX_NAV BIT(15) +#define B_BE_ACK_BA_RESP_HE_CHK_INTRA_NAV BIT(14) +#define B_BE_ACK_BA_RESP_HE_CHK_BASIC_NAV BIT(13) +#define B_BE_ACK_BA_RESP_HE_CHK_BTCCA BIT(12) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_EDCCA160 BIT(11) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_EDCCA80 BIT(10) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_EDCCA40 BIT(9) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_EDCCA20 BIT(8) +#define B_BE_ACK_BA_RESP_HE_CHK_EDCCA_PER20_BMP BIT(7) +#define B_BE_ACK_BA_RESP_HE_CHK_CCA_PER20_BMP BIT(6) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_CCA160 BIT(5) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_CCA80 BIT(4) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_CCA40 BIT(3) +#define B_BE_ACK_BA_RESP_HE_CHK_SEC_CCA20 BIT(2) +#define B_BE_ACK_BA_RESP_HE_CHK_EDCCA BIT(1) +#define B_BE_ACK_BA_RESP_HE_CHK_CCA BIT(0) + +#define R_BE_WMAC_ACK_BA_RESP_EHT_LEG_PUNC 0x11208 +#define R_BE_WMAC_ACK_BA_RESP_EHT_LEG_PUNC_C1 0x15208 +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_NSTR BIT(16) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_TX_NAV BIT(15) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_INTRA_NAV BIT(14) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_BASIC_NAV BIT(13) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_BTCCA BIT(12) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_EDCCA160 BIT(11) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_EDCCA80 BIT(10) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_EDCCA40 BIT(9) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_EDCCA20 BIT(8) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_EDCCA_PER20_BMP BIT(7) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_CCA_PER20_BMP BIT(6) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_CCA160 BIT(5) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_CCA80 BIT(4) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_CCA40 BIT(3) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_SEC_CCA20 BIT(2) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_EDCCA BIT(1) +#define B_BE_ACK_BA_EHT_LEG_PUNC_CHK_CCA BIT(0) + +#define R_BE_RCR 0x11400 +#define R_BE_RCR_C1 0x15400 +#define B_BE_BUSY_CHKSN BIT(15) +#define B_BE_DYN_CHEN BIT(14) +#define B_BE_AUTO_RST BIT(13) +#define B_BE_TIMER_SEL BIT(12) +#define B_BE_STOP_RX_IN BIT(11) +#define B_BE_PSR_RDY_CHKDIS BIT(10) +#define B_BE_DRV_INFO_SZ_MASK GENMASK(9, 8) +#define B_BE_HDR_CNV_SZ_MASK GENMASK(7, 6) +#define B_BE_PHY_RPT_SZ_MASK GENMASK(5, 4) +#define B_BE_CH_EN BIT(0) + +#define R_BE_DLK_PROTECT_CTL 0x11402 +#define R_BE_DLK_PROTECT_CTL_C1 0x15402 +#define B_BE_RX_DLK_CCA_TIME_MASK GENMASK(15, 8) +#define TRXCFG_RMAC_CCA_TO 32 +#define B_BE_RX_DLK_DATA_TIME_MASK GENMASK(7, 4) +#define TRXCFG_RMAC_DATA_TO 15 +#define B_BE_RX_DLK_RST_FSM BIT(3) +#define B_BE_RX_DLK_RST_SKIPDMA BIT(2) +#define B_BE_RX_DLK_RST_EN BIT(1) +#define B_BE_RX_DLK_INT_EN BIT(0) + +#define R_BE_PLCP_HDR_FLTR 0x11404 +#define R_BE_PLCP_HDR_FLTR_C1 0x15404 +#define B_BE_PLCP_RXFA_RESET_TYPE_MASK GENMASK(15, 12) +#define B_BE_PLCP_RXFA_RESET_EN BIT(11) +#define B_BE_DIS_CHK_MIN_LEN BIT(8) +#define B_BE_HE_SIGB_CRC_CHK BIT(6) +#define B_BE_VHT_MU_SIGB_CRC_CHK BIT(5) +#define B_BE_VHT_SU_SIGB_CRC_CHK BIT(4) +#define B_BE_SIGA_CRC_CHK BIT(3) +#define B_BE_LSIG_PARITY_CHK_EN BIT(2) +#define B_BE_CCK_SIG_CHK BIT(1) +#define B_BE_CCK_CRC_CHK BIT(0) + #define R_BE_RX_FLTR_OPT 0x11420 #define R_BE_RX_FLTR_OPT_C1 0x15420 #define B_BE_UID_FILTER_MASK GENMASK(31, 24) @@ -6545,6 +7014,55 @@ #define B_BE_A_A1_MATCH BIT(1) #define B_BE_SNIFFER_MODE BIT(0) +#define R_BE_CTRL_FLTR 0x11424 +#define R_BE_CTRL_FLTR_C1 0x15424 +#define B_BE_CTRL_STYPE_MASK GENMASK(15, 0) +#define RX_FLTR_FRAME_DROP_BE 0x0000 +#define RX_FLTR_FRAME_ACCEPT_BE 0xFFFF + +#define R_BE_MGNT_FLTR 0x11428 +#define R_BE_MGNT_FLTR_C1 0x15428 +#define B_BE_MGNT_STYPE_MASK GENMASK(15, 0) + +#define R_BE_DATA_FLTR 0x1142C +#define R_BE_DATA_FLTR_C1 0x1542C +#define B_BE_DATA_STYPE_MASK GENMASK(15, 0) + +#define R_BE_ADDR_CAM_CTRL 0x11434 +#define R_BE_ADDR_CAM_CTRL_C1 0x15434 +#define B_BE_ADDR_CAM_RANGE_MASK GENMASK(23, 16) +#define ADDR_CAM_SERCH_RANGE 0x7f +#define B_BE_ADDR_CAM_CMPLIMT_MASK GENMASK(15, 12) +#define B_BE_ADDR_CAM_IORST BIT(10) +#define B_BE_DIS_ADDR_CLK_GATED BIT(9) +#define B_BE_ADDR_CAM_CLR BIT(8) +#define B_BE_ADDR_CAM_A2_B0_CHK BIT(2) +#define B_BE_ADDR_CAM_SRCH_PERPKT BIT(1) +#define B_BE_ADDR_CAM_EN BIT(0) + +#define R_BE_RESPBA_CAM_CTRL 0x1143C +#define R_BE_RESPBA_CAM_CTRL_C1 0x1543C +#define B_BE_BACAM_SKIP_ALL_QOSNULL BIT(24) +#define B_BE_BACAM_STD_SSN_SEL BIT(20) +#define B_BE_BACAM_TEMP_SZ_MASK GENMASK(17, 16) +#define B_BE_BACAM_RST_IDX_MASK GENMASK(15, 8) +#define B_BE_BACAM_SHIFT_POLL BIT(7) +#define B_BE_BACAM_IORST BIT(6) +#define B_BE_BACAM_GCK_DIS BIT(5) +#define B_BE_COMPL_VAL BIT(3) +#define B_BE_SSN_SEL BIT(2) +#define B_BE_BACAM_RST_MASK GENMASK(1, 0) +#define S_BE_BACAM_RST_DONE 0 +#define S_BE_BACAM_RST_ENT 1 +#define S_BE_BACAM_RST_ALL 2 + +#define R_BE_RX_SR_CTRL 0x1144A +#define R_BE_RX_SR_CTRL_C1 0x1544A +#define B_BE_SR_OP_MODE_MASK GENMASK(5, 4) +#define B_BE_SRG_CHK_EN BIT(2) +#define B_BE_SR_CTRL_PLCP_EN BIT(1) +#define B_BE_SR_EN BIT(0) + #define R_BE_CSIRPT_OPTION 0x11464 #define R_BE_CSIRPT_OPTION_C1 0x15464 #define B_BE_CSIPRT_EHTSU_AID_EN BIT(26) @@ -6590,6 +7108,29 @@ B_BE_RX_ERR_STS_ACT_TO_MSK | \ B_BE_RX_ERR_TRIG_ACT_TO_MSK) +#define R_BE_RX_PLCP_EXT_OPTION_1 0x11514 +#define R_BE_RX_PLCP_EXT_OPTION_1_C1 0x15514 +#define B_BE_PLCP_CLOSE_RX_UNSPUUORT BIT(19) +#define B_BE_PLCP_CLOSE_RX_BB_BRK BIT(18) +#define B_BE_PLCP_CLOSE_RX_PSDU_PRES BIT(17) +#define B_BE_PLCP_CLOSE_RX_NDP BIT(16) +#define B_BE_PLCP_NSS_SRC BIT(11) +#define B_BE_PLCP_DOPPLEB_BE_SRC BIT(10) +#define B_BE_PLCP_STBC_SRC BIT(9) +#define B_BE_PLCP_SU_PSDU_LEN_SRC BIT(8) +#define B_BE_PLCP_RXSB_SRC BIT(7) +#define B_BE_PLCP_BW_SRC_MASK GENMASK(6, 5) +#define B_BE_PLCP_GILTF_SRC BIT(4) +#define B_BE_PLCP_NSTS_SRC BIT(3) +#define B_BE_PLCP_MCS_SRC BIT(2) +#define B_BE_PLCP_CH20_WIDATA_SRC BIT(1) +#define B_BE_PLCP_PPDU_TYPE_SRC BIT(0) + +#define R_BE_RESP_CSI_RESERVED_PAGE 0x11810 +#define R_BE_RESP_CSI_RESERVED_PAGE_C1 0x15810 +#define B_BE_CSI_RESERVED_PAGE_NUM_MASK GENMASK(27, 16) +#define B_BE_CSI_RESERVED_START_PAGE_MASK GENMASK(11, 0) + #define R_BE_RESP_IMR 0x11884 #define R_BE_RESP_IMR_C1 0x15884 #define B_BE_RESP_TBL_FLAG_ERR_ISR_EN BIT(17) diff --git a/drivers/net/wireless/virtual/mac80211_hwsim.c b/drivers/net/wireless/virtual/mac80211_hwsim.c index c7b4414cc6c3..a84340c2075f 100644 --- a/drivers/net/wireless/virtual/mac80211_hwsim.c +++ b/drivers/net/wireless/virtual/mac80211_hwsim.c @@ -190,10 +190,25 @@ static const struct ieee80211_regdomain hwsim_world_regdom_custom_03 = { } }; +static const struct ieee80211_regdomain hwsim_world_regdom_custom_04 = { + .n_reg_rules = 6, + .alpha2 = "99", + .reg_rules = { + REG_RULE(2412 - 10, 2462 + 10, 40, 0, 20, 0), + REG_RULE(2484 - 10, 2484 + 10, 40, 0, 20, 0), + REG_RULE(5150 - 10, 5240 + 10, 80, 0, 30, 0), + REG_RULE(5260 - 10, 5320 + 10, 80, 0, 30, + NL80211_RRF_DFS_CONCURRENT | NL80211_RRF_DFS), + REG_RULE(5745 - 10, 5825 + 10, 80, 0, 30, 0), + REG_RULE(5855 - 10, 5925 + 10, 80, 0, 33, 0), + } +}; + static const struct ieee80211_regdomain *hwsim_world_regdom_custom[] = { &hwsim_world_regdom_custom_01, &hwsim_world_regdom_custom_02, &hwsim_world_regdom_custom_03, + &hwsim_world_regdom_custom_04, }; struct hwsim_vif_priv { @@ -4029,6 +4044,8 @@ static const struct ieee80211_sband_iftype_data sband_capa_2ghz[] = { IEEE80211_HE_MAC_CAP3_OMI_CONTROL | IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[0] = + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_IN_2G, .phy_cap_info[1] = IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | @@ -4134,6 +4151,8 @@ static const struct ieee80211_sband_iftype_data sband_capa_2ghz[] = { IEEE80211_HE_MAC_CAP3_OMI_CONTROL | IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[0] = + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_IN_2G, .phy_cap_info[1] = IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | @@ -4237,6 +4256,8 @@ static const struct ieee80211_sband_iftype_data sband_capa_2ghz[] = { IEEE80211_HE_MAC_CAP3_OMI_CONTROL | IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[0] = + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_IN_2G, .phy_cap_info[1] = IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | @@ -5288,6 +5309,10 @@ static int mac80211_hwsim_new_radio(struct genl_info *info, schedule_timeout_interruptible(1); } + /* TODO: Add param */ + wiphy_ext_feature_set(hw->wiphy, + NL80211_EXT_FEATURE_DFS_CONCURRENT); + if (param->no_vif) ieee80211_hw_set(hw, NO_AUTO_VIF); diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index b022af3d20fe..bf525ef32209 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -1716,20 +1716,6 @@ ptp_ocp_get_mem(struct ptp_ocp *bp, struct ocp_resource *r) return __ptp_ocp_get_mem(bp, start, r->size); } -static void -ptp_ocp_set_irq_resource(struct resource *res, int irq) -{ - struct resource r = DEFINE_RES_IRQ(irq); - *res = r; -} - -static void -ptp_ocp_set_mem_resource(struct resource *res, resource_size_t start, int size) -{ - struct resource r = DEFINE_RES_MEM(start, size); - *res = r; -} - static int ptp_ocp_register_spi(struct ptp_ocp *bp, struct ocp_resource *r) { @@ -1741,15 +1727,15 @@ ptp_ocp_register_spi(struct ptp_ocp *bp, struct ocp_resource *r) int id; start = pci_resource_start(pdev, 0) + r->offset; - ptp_ocp_set_mem_resource(&res[0], start, r->size); - ptp_ocp_set_irq_resource(&res[1], pci_irq_vector(pdev, r->irq_vec)); + res[0] = DEFINE_RES_MEM(start, r->size); + res[1] = DEFINE_RES_IRQ(pci_irq_vector(pdev, r->irq_vec)); info = r->extra; id = pci_dev_id(pdev) << 1; id += info->pci_offset; p = platform_device_register_resndata(&pdev->dev, info->name, id, - res, 2, info->data, + res, ARRAY_SIZE(res), info->data, info->data_size); if (IS_ERR(p)) return PTR_ERR(p); @@ -1768,11 +1754,11 @@ ptp_ocp_i2c_bus(struct pci_dev *pdev, struct ocp_resource *r, int id) info = r->extra; start = pci_resource_start(pdev, 0) + r->offset; - ptp_ocp_set_mem_resource(&res[0], start, r->size); - ptp_ocp_set_irq_resource(&res[1], pci_irq_vector(pdev, r->irq_vec)); + res[0] = DEFINE_RES_MEM(start, r->size); + res[1] = DEFINE_RES_IRQ(pci_irq_vector(pdev, r->irq_vec)); return platform_device_register_resndata(&pdev->dev, info->name, - id, res, 2, + id, res, ARRAY_SIZE(res), info->data, info->data_size); } diff --git a/drivers/s390/net/ism.h b/drivers/s390/net/ism.h index 70c5bbda0fea..047fa6101555 100644 --- a/drivers/s390/net/ism.h +++ b/drivers/s390/net/ism.h @@ -16,7 +16,6 @@ */ #define ISM_DMB_WORD_OFFSET 1 #define ISM_DMB_BIT_OFFSET (ISM_DMB_WORD_OFFSET * 32) -#define ISM_IDENT_MASK 0x00FFFF #define ISM_REG_SBA 0x1 #define ISM_REG_IEQ 0x2 @@ -192,12 +191,6 @@ struct ism_sba { #define ISM_CREATE_REQ(dmb, idx, sf, offset) \ ((dmb) | (idx) << 24 | (sf) << 23 | (offset)) -struct ism_systemeid { - u8 seid_string[24]; - u8 serial_number[4]; - u8 type[4]; -}; - static inline void __ism_read_cmd(struct ism_dev *ism, void *data, unsigned long offset, unsigned long len) { diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index 81aabbfbbe2c..2c8e964425dc 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -36,6 +36,7 @@ static struct ism_client *clients[MAX_CLIENTS]; /* use an array rather than */ /* a list for fast mapping */ static u8 max_client; static DEFINE_MUTEX(clients_lock); +static bool ism_v2_capable; struct ism_dev_list { struct list_head list; struct mutex mutex; /* protects ism device list */ @@ -443,32 +444,6 @@ int ism_move(struct ism_dev *ism, u64 dmb_tok, unsigned int idx, bool sf, } EXPORT_SYMBOL_GPL(ism_move); -static struct ism_systemeid SYSTEM_EID = { - .seid_string = "IBM-SYSZ-ISMSEID00000000", - .serial_number = "0000", - .type = "0000", -}; - -static void ism_create_system_eid(void) -{ - struct cpuid id; - u16 ident_tail; - char tmp[5]; - - get_cpu_id(&id); - ident_tail = (u16)(id.ident & ISM_IDENT_MASK); - snprintf(tmp, 5, "%04X", ident_tail); - memcpy(&SYSTEM_EID.serial_number, tmp, 4); - snprintf(tmp, 5, "%04X", id.machine); - memcpy(&SYSTEM_EID.type, tmp, 4); -} - -u8 *ism_get_seid(void) -{ - return SYSTEM_EID.seid_string; -} -EXPORT_SYMBOL_GPL(ism_get_seid); - static void ism_handle_event(struct ism_dev *ism) { struct ism_event *entry; @@ -560,7 +535,9 @@ static int ism_dev_init(struct ism_dev *ism) if (!ism_add_vlan_id(ism, ISM_RESERVED_VLANID)) /* hardware is V2 capable */ - ism_create_system_eid(); + ism_v2_capable = true; + else + ism_v2_capable = false; mutex_lock(&ism_dev_list.mutex); mutex_lock(&clients_lock); @@ -665,8 +642,7 @@ static void ism_dev_exit(struct ism_dev *ism) } mutex_unlock(&clients_lock); - if (SYSTEM_EID.serial_number[0] != '0' || - SYSTEM_EID.type[0] != '0') + if (ism_v2_capable) ism_del_vlan_id(ism, ISM_RESERVED_VLANID); unregister_ieq(ism); unregister_sba(ism); @@ -743,10 +719,10 @@ static int ism_query_rgid(struct ism_dev *ism, u64 rgid, u32 vid_valid, return ism_cmd(ism, &cmd); } -static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid, - u32 vid) +static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) { - return ism_query_rgid(smcd->priv, rgid, vid_valid, vid); + return ism_query_rgid(smcd->priv, rgid->gid, vid_valid, vid); } static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, @@ -797,10 +773,11 @@ static int ism_signal_ieq(struct ism_dev *ism, u64 rgid, u32 trigger_irq, return ism_cmd(ism, &cmd); } -static int smcd_signal_ieq(struct smcd_dev *smcd, u64 rgid, u32 trigger_irq, - u32 event_code, u64 info) +static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info) { - return ism_signal_ieq(smcd->priv, rgid, trigger_irq, event_code, info); + return ism_signal_ieq(smcd->priv, rgid->gid, + trigger_irq, event_code, info); } static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, @@ -812,8 +789,7 @@ static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, static int smcd_supports_v2(void) { - return SYSTEM_EID.serial_number[0] != '0' || - SYSTEM_EID.type[0] != '0'; + return ism_v2_capable; } static u64 ism_get_local_gid(struct ism_dev *ism) @@ -821,9 +797,11 @@ static u64 ism_get_local_gid(struct ism_dev *ism) return ism->local_gid; } -static u64 smcd_get_local_gid(struct smcd_dev *smcd) +static void smcd_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) { - return ism_get_local_gid(smcd->priv); + smcd_gid->gid = ism_get_local_gid(smcd->priv); + smcd_gid->gid_ext = 0; } static u16 ism_get_chid(struct ism_dev *ism) @@ -857,7 +835,6 @@ static const struct smcd_ops ism_ops = { .signal_event = smcd_signal_ieq, .move_data = smcd_move, .supports_v2 = smcd_supports_v2, - .get_system_eid = ism_get_seid, .get_local_gid = smcd_get_local_gid, .get_chid = smcd_get_chid, .get_dev = smcd_get_dev, diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 7a8d4c81a39a..7671530d6e4e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2466,12 +2466,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, struct btf_func_model *m); struct bpf_reg_state; -int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *regs); -int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *regs); -int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *reg, u32 *nargs); +int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog); int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog, struct btf *btf, const struct btf_type *t); const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt, diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index c2819a6579a5..d07d857ca67f 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -606,6 +606,13 @@ static inline bool bpf_verifier_log_needed(const struct bpf_verifier_log *log) #define BPF_MAX_SUBPROGS 256 +struct bpf_subprog_arg_info { + enum bpf_arg_type arg_type; + union { + u32 mem_size; + }; +}; + struct bpf_subprog_info { /* 'start' has to be the first field otherwise find_subprog() won't work */ u32 start; /* insn idx of function entry point */ @@ -617,6 +624,10 @@ struct bpf_subprog_info { bool is_cb: 1; bool is_async_cb: 1; bool is_exception_cb: 1; + bool args_cached: 1; + + u8 arg_cnt; + struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS]; }; struct bpf_verifier_env; @@ -727,6 +738,16 @@ struct bpf_verifier_env { char tmp_str_buf[TMP_STR_BUF_LEN]; }; +static inline struct bpf_func_info_aux *subprog_aux(struct bpf_verifier_env *env, int subprog) +{ + return &env->prog->aux->func_info_aux[subprog]; +} + +static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env, int subprog) +{ + return &env->subprog_info[subprog]; +} + __printf(2, 0) void bpf_verifier_vlog(struct bpf_verifier_log *log, const char *fmt, va_list args); __printf(2, 3) void bpf_verifier_log_write(struct bpf_verifier_env *env, @@ -764,14 +785,6 @@ bpf_prog_offload_replace_insn(struct bpf_verifier_env *env, u32 off, void bpf_prog_offload_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt); -int check_ptr_off_reg(struct bpf_verifier_env *env, - const struct bpf_reg_state *reg, int regno); -int check_func_arg_reg_off(struct bpf_verifier_env *env, - const struct bpf_reg_state *reg, int regno, - enum bpf_arg_type arg_type); -int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno, u32 mem_size); - /* this lives here instead of in bpf.h because it needs to dereference tgt_prog */ static inline u64 bpf_trampoline_compute_key(const struct bpf_prog *tgt_prog, struct btf *btf, u32 btf_id) diff --git a/include/linux/ethtool.h b/include/linux/ethtool.h index cfcd952a1d4f..325e0778e937 100644 --- a/include/linux/ethtool.h +++ b/include/linux/ethtool.h @@ -411,8 +411,10 @@ struct ethtool_pause_stats { * not entire FEC data blocks. This is a non-standard statistic. * Reported to user space as %ETHTOOL_A_FEC_STAT_CORR_BITS. * - * @lane: per-lane/PCS-instance counts as defined by the standard - * @total: error counts for the entire port, for drivers incapable of reporting + * For each of the above fields, the two substructure members are: + * + * - @lanes: per-lane/PCS-instance counts as defined by the standard + * - @total: error counts for the entire port, for drivers incapable of reporting * per-lane stats * * Drivers should fill in either only total or per-lane statistics, core diff --git a/include/linux/ieee80211.h b/include/linux/ieee80211.h index 52808dd22513..83c4d060a559 100644 --- a/include/linux/ieee80211.h +++ b/include/linux/ieee80211.h @@ -2720,6 +2720,7 @@ static inline bool ieee80211_he_capa_size_ok(const u8 *data, u8 len) #define IEEE80211_6GHZ_CTRL_REG_LPI_AP 0 #define IEEE80211_6GHZ_CTRL_REG_SP_AP 1 +#define IEEE80211_6GHZ_CTRL_REG_VLP_AP 2 /** * struct ieee80211_he_6ghz_oper - HE 6 GHz operation Information field diff --git a/include/linux/ism.h b/include/linux/ism.h index 9a4c204df3da..5428edd90982 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -86,7 +86,6 @@ int ism_register_dmb(struct ism_dev *dev, struct ism_dmb *dmb, int ism_unregister_dmb(struct ism_dev *dev, struct ism_dmb *dmb); int ism_move(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); -u8 *ism_get_seid(void); const struct smcd_ops *ism_get_smcd_ops(void); diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 7ee5b79ff3d6..2bba88c67f58 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -681,6 +681,7 @@ struct mlx5e_resources { struct mlx5_sq_bfreg bfreg; #define MLX5_MAX_NUM_TC 8 u32 tisn[MLX5_MAX_PORTS][MLX5_MAX_NUM_TC]; + bool tisn_valid; } hw_objs; struct net_device *uplink_netdev; struct mutex uplink_netdev_lock; @@ -821,6 +822,7 @@ struct mlx5_core_dev { struct blocking_notifier_head macsec_nh; #endif u64 num_ipsec_offloads; + struct mlx5_sd *sd; }; struct mlx5_db { @@ -1222,6 +1224,14 @@ static inline bool mlx5_core_is_ecpf(const struct mlx5_core_dev *dev) return dev->caps.embedded_cpu; } +static inline bool mlx5_core_is_mgmt_pf(const struct mlx5_core_dev *dev) +{ + if (!MLX5_CAP_GEN_2(dev, local_mng_port_valid)) + return false; + + return MLX5_CAP_GEN_2(dev, local_mng_port); +} + static inline bool mlx5_core_is_ecpf_esw_manager(const struct mlx5_core_dev *dev) { diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index fee20fc010c2..586569209254 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -1954,8 +1954,10 @@ enum { struct mlx5_ifc_cmd_hca_cap_2_bits { u8 reserved_at_0[0x80]; - u8 migratable[0x1]; - u8 reserved_at_81[0x1f]; + u8 migratable[0x1]; + u8 reserved_at_81[0x19]; + u8 local_mng_port[0x1]; + u8 reserved_at_9b[0x5]; u8 max_reformat_insert_size[0x8]; u8 max_reformat_insert_offset[0x8]; @@ -1973,7 +1975,13 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 allowed_object_for_other_vhca_access[0x40]; - u8 reserved_at_140[0x60]; + u8 reserved_at_140[0x20]; + + u8 reserved_at_160[0xa]; + u8 local_mng_port_valid[0x1]; + u8 reserved_at_16b[0x15]; + + u8 reserved_at_180[0x20]; u8 flow_table_type_2_type[0x8]; u8 reserved_at_1a8[0x3]; @@ -4030,8 +4038,13 @@ struct mlx5_ifc_nic_vport_context_bits { u8 affiliation_criteria[0x4]; u8 affiliated_vhca_id[0x10]; - u8 reserved_at_60[0xd0]; + u8 reserved_at_60[0xa0]; + u8 reserved_at_100[0x1]; + u8 sd_group[0x3]; + u8 reserved_at_104[0x1c]; + + u8 reserved_at_120[0x10]; u8 mtu[0x10]; u8 system_image_guid[0x40]; @@ -10116,8 +10129,7 @@ struct mlx5_ifc_mpir_reg_bits { u8 reserved_at_20[0x20]; u8 local_port[0x8]; - u8 reserved_at_28[0x15]; - u8 sd_group[0x3]; + u8 reserved_at_28[0x18]; u8 reserved_at_60[0x20]; }; diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h index fbb9bf447889..c36cc6d82926 100644 --- a/include/linux/mlx5/vport.h +++ b/include/linux/mlx5/vport.h @@ -72,6 +72,7 @@ int mlx5_query_nic_vport_mtu(struct mlx5_core_dev *mdev, u16 *mtu); int mlx5_modify_nic_vport_mtu(struct mlx5_core_dev *mdev, u16 mtu); int mlx5_query_nic_vport_system_image_guid(struct mlx5_core_dev *mdev, u64 *system_image_guid); +int mlx5_query_nic_vport_sd_group(struct mlx5_core_dev *mdev, u8 *sd_group); int mlx5_query_nic_vport_node_guid(struct mlx5_core_dev *mdev, u64 *node_guid); int mlx5_modify_nic_vport_node_guid(struct mlx5_core_dev *mdev, u16 vport, u64 node_guid); diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 75c7725e5e4f..d59db9adcc96 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -40,7 +40,6 @@ #include <net/dcbnl.h> #endif #include <net/netprio_cgroup.h> - #include <linux/netdev_features.h> #include <linux/neighbour.h> #include <uapi/linux/netdevice.h> @@ -52,6 +51,7 @@ #include <net/net_trackers.h> #include <net/net_debug.h> #include <net/dropreason-core.h> +#include <linux/phy_link_topology_core.h> struct netpoll_info; struct device; @@ -2047,6 +2047,7 @@ enum netdev_stat_type { * @fcoe_ddp_xid: Max exchange id for FCoE LRO by ddp * * @priomap: XXX: need comments on this one + * @link_topo: Physical link topology tracking attached PHYs * @phydev: Physical device may attach itself * for hardware timestamping * @sfp_bus: attached &struct sfp_bus structure. @@ -2114,6 +2115,7 @@ struct net_device { const struct net_device_ops *netdev_ops; const struct header_ops *header_ops; struct netdev_queue *_tx; + netdev_features_t gso_partial_features; unsigned int real_num_tx_queues; unsigned int gso_max_size; unsigned int gso_ipv4_max_size; @@ -2210,7 +2212,6 @@ struct net_device { netdev_features_t vlan_features; netdev_features_t hw_enc_features; netdev_features_t mpls_features; - netdev_features_t gso_partial_features; unsigned int min_mtu; unsigned int max_mtu; @@ -2441,6 +2442,7 @@ struct net_device { #if IS_ENABLED(CONFIG_CGROUP_NET_PRIO) struct netprio_map __rcu *priomap; #endif + struct phy_link_topology link_topo; struct phy_device *phydev; struct sfp_bus *sfp_bus; struct lock_class_key *qdisc_tx_busylock; diff --git a/include/linux/phy.h b/include/linux/phy.h index e9e85d347587..ac22b8e28a85 100644 --- a/include/linux/phy.h +++ b/include/linux/phy.h @@ -434,6 +434,9 @@ struct mii_bus { /** @shared: shared state across different PHYs */ struct phy_package_shared *shared[PHY_MAX_ADDR]; + + /** @__unregister_callback: called at the last step of unregistration */ + void (*__unregister_callback)(struct mii_bus *bus); }; #define to_mii_bus(d) container_of(d, struct mii_bus, dev) @@ -544,6 +547,9 @@ struct macsec_ops; * @drv: Pointer to the driver for this PHY instance * @devlink: Create a link between phy dev and mac dev, if the external phy * used by current mac interface is managed by another mac interface. + * @phyindex: Unique id across the phy's parent tree of phys to address the PHY + * from userspace, similar to ifindex. A zero index means the PHY + * wasn't assigned an id yet. * @phy_id: UID for this device found during discovery * @c45_ids: 802.3-c45 Device Identifiers if is_c45. * @is_c45: Set to true if this PHY uses clause 45 addressing. @@ -643,6 +649,7 @@ struct phy_device { struct device_link *devlink; + u32 phyindex; u32 phy_id; struct phy_c45_device_ids c45_ids; @@ -1722,6 +1729,8 @@ int phy_suspend(struct phy_device *phydev); int phy_resume(struct phy_device *phydev); int __phy_resume(struct phy_device *phydev); int phy_loopback(struct phy_device *phydev, bool enable); +int phy_sfp_connect_phy(void *upstream, struct phy_device *phy); +void phy_sfp_disconnect_phy(void *upstream, struct phy_device *phy); void phy_sfp_attach(void *upstream, struct sfp_bus *bus); void phy_sfp_detach(void *upstream, struct sfp_bus *bus); int phy_sfp_probe(struct phy_device *phydev, diff --git a/include/linux/phy_link_topology.h b/include/linux/phy_link_topology.h new file mode 100644 index 000000000000..91902263ec0e --- /dev/null +++ b/include/linux/phy_link_topology.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * PHY device list allow maintaining a list of PHY devices that are + * part of a netdevice's link topology. PHYs can for example be chained, + * as is the case when using a PHY that exposes an SFP module, on which an + * SFP transceiver that embeds a PHY is connected. + * + * This list can then be used by userspace to leverage individual PHY + * capabilities. + */ +#ifndef __PHY_LINK_TOPOLOGY_H +#define __PHY_LINK_TOPOLOGY_H + +#include <linux/ethtool.h> +#include <linux/phy_link_topology_core.h> + +struct xarray; +struct phy_device; +struct net_device; +struct sfp_bus; + +struct phy_device_node { + enum phy_upstream upstream_type; + + union { + struct net_device *netdev; + struct phy_device *phydev; + } upstream; + + struct sfp_bus *parent_sfp_bus; + + struct phy_device *phy; +}; + +static inline struct phy_device * +phy_link_topo_get_phy(struct phy_link_topology *topo, u32 phyindex) +{ + struct phy_device_node *pdn = xa_load(&topo->phys, phyindex); + + if (pdn) + return pdn->phy; + + return NULL; +} + +#if IS_ENABLED(CONFIG_PHYLIB) +int phy_link_topo_add_phy(struct phy_link_topology *topo, + struct phy_device *phy, + enum phy_upstream upt, void *upstream); + +void phy_link_topo_del_phy(struct phy_link_topology *lt, struct phy_device *phy); + +#else +static inline int phy_link_topo_add_phy(struct phy_link_topology *topo, + struct phy_device *phy, + enum phy_upstream upt, void *upstream) +{ + return 0; +} + +static inline void phy_link_topo_del_phy(struct phy_link_topology *topo, + struct phy_device *phy) +{ +} +#endif + +#endif /* __PHY_LINK_TOPOLOGY_H */ diff --git a/include/linux/phy_link_topology_core.h b/include/linux/phy_link_topology_core.h new file mode 100644 index 000000000000..78c75f909489 --- /dev/null +++ b/include/linux/phy_link_topology_core.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __PHY_LINK_TOPOLOGY_CORE_H +#define __PHY_LINK_TOPOLOGY_CORE_H + +struct xarray; + +struct phy_link_topology { + struct xarray phys; + + u32 next_phy_index; +}; + +static inline void phy_link_topo_init(struct phy_link_topology *topo) +{ + xa_init_flags(&topo->phys, XA_FLAGS_ALLOC1); + topo->next_phy_index = 1; +} + +#endif /* __PHY_LINK_TOPOLOGY_CORE_H */ diff --git a/include/linux/sfp.h b/include/linux/sfp.h index 9346cd44814d..55c0ab17c9e2 100644 --- a/include/linux/sfp.h +++ b/include/linux/sfp.h @@ -544,7 +544,7 @@ struct sfp_upstream_ops { void (*link_down)(void *priv); void (*link_up)(void *priv); int (*connect_phy)(void *priv, struct phy_device *); - void (*disconnect_phy)(void *priv); + void (*disconnect_phy)(void *priv, struct phy_device *); }; #if IS_ENABLED(CONFIG_SFP) @@ -570,6 +570,7 @@ struct sfp_bus *sfp_bus_find_fwnode(const struct fwnode_handle *fwnode); int sfp_bus_add_upstream(struct sfp_bus *bus, void *upstream, const struct sfp_upstream_ops *ops); void sfp_bus_del_upstream(struct sfp_bus *bus); +const char *sfp_get_name(struct sfp_bus *bus); #else static inline int sfp_parse_port(struct sfp_bus *bus, const struct sfp_eeprom_id *id, @@ -648,6 +649,11 @@ static inline int sfp_bus_add_upstream(struct sfp_bus *bus, void *upstream, static inline void sfp_bus_del_upstream(struct sfp_bus *bus) { } + +static inline const char *sfp_get_name(struct sfp_bus *bus) +{ + return NULL; +} #endif #endif diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index ea5c8ab3ed00..a5ae952454c8 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -754,7 +754,6 @@ typedef unsigned char *sk_buff_data_t; * @dev_scratch: (aka @dev) alternate use of @dev when @dev would be %NULL * @cb: Control buffer. Free for use by every layer. Put private vars here * @_skb_refdst: destination entry (with norefcount bit) - * @sp: the security path, used for xfrm * @len: Length of actual data * @data_len: Data length * @mac_len: Length of link layer header @@ -788,7 +787,6 @@ typedef unsigned char *sk_buff_data_t; * @tcp_tsorted_anchor: list structure for TCP (tp->tsorted_sent_queue) * @_sk_redir: socket redirection information for skmsg * @_nfct: Associated connection, if any (with nfctinfo bits) - * @nf_bridge: Saved data about a bridged frame - see br_netfilter.c * @skb_iif: ifindex of device we arrived on * @tc_index: Traffic control index * @hash: the packet hash @@ -4009,6 +4007,7 @@ struct sk_buff *skb_segment_list(struct sk_buff *skb, netdev_features_t features unsigned int offset); struct sk_buff *skb_vlan_untag(struct sk_buff *skb); int skb_ensure_writable(struct sk_buff *skb, unsigned int write_len); +int skb_ensure_writable_head_tail(struct sk_buff *skb, struct net_device *dev); int __skb_vlan_pop(struct sk_buff *skb, u16 *vlan_tci); int skb_vlan_pop(struct sk_buff *skb); int skb_vlan_push(struct sk_buff *skb, __be16 vlan_proto, u16 vlan_tci); diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index ac1fb326dcda..cf79656ce09c 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -117,6 +117,11 @@ struct wiphy; * This may be due to the driver or due to regulatory bandwidth * restrictions. * @IEEE80211_CHAN_NO_EHT: EHT operation is not permitted on this channel. + * @IEEE80211_CHAN_DFS_CONCURRENT: See %NL80211_RRF_DFS_CONCURRENT + * @IEEE80211_CHAN_NO_UHB_VLP_CLIENT: Client connection with VLP AP + * not permitted using this channel + * @IEEE80211_CHAN_NO_UHB_AFC_CLIENT: Client connection with AFC AP + * not permitted using this channel */ enum ieee80211_channel_flags { IEEE80211_CHAN_DISABLED = 1<<0, @@ -140,6 +145,9 @@ enum ieee80211_channel_flags { IEEE80211_CHAN_16MHZ = 1<<18, IEEE80211_CHAN_NO_320MHZ = 1<<19, IEEE80211_CHAN_NO_EHT = 1<<20, + IEEE80211_CHAN_DFS_CONCURRENT = 1<<21, + IEEE80211_CHAN_NO_UHB_VLP_CLIENT= 1<<22, + IEEE80211_CHAN_NO_UHB_AFC_CLIENT= 1<<23, }; #define IEEE80211_CHAN_NO_HT40 \ @@ -3225,8 +3233,8 @@ struct cfg80211_ibss_params { * * @behaviour: requested BSS selection behaviour. * @param: parameters for requestion behaviour. - * @band_pref: preferred band for %NL80211_BSS_SELECT_ATTR_BAND_PREF. - * @adjust: parameters for %NL80211_BSS_SELECT_ATTR_RSSI_ADJUST. + * @param.band_pref: preferred band for %NL80211_BSS_SELECT_ATTR_BAND_PREF. + * @param.adjust: parameters for %NL80211_BSS_SELECT_ATTR_RSSI_ADJUST. */ struct cfg80211_bss_selection { enum nl80211_bss_select_attr behaviour; @@ -6063,7 +6071,6 @@ void wiphy_delayed_work_flush(struct wiphy *wiphy, * wireless device if it has no netdev * @u: union containing data specific to @iftype * @connected: indicates if connected or not (STA mode) - * @bssid: (private) Used by the internal configuration code * @wext: (private) Used by the internal wireless extensions compat code * @wext.ibss: (private) IBSS data part of wext handling * @wext.connect: (private) connection handling data @@ -6083,8 +6090,6 @@ void wiphy_delayed_work_flush(struct wiphy *wiphy, * @mgmt_registrations: list of registrations for management frames * @mgmt_registrations_need_update: mgmt registrations were updated, * need to propagate the update to the driver - * @beacon_interval: beacon interval used on this device for transmitting - * beacons, 0 when not valid * @address: The address for this device, valid only if @netdev is %NULL * @is_running: true if this is a non-netdev device that has been started, e.g. * the P2P Device. @@ -7166,6 +7171,23 @@ int cfg80211_get_ies_channel_number(const u8 *ie, size_t ielen, enum nl80211_band band); /** + * cfg80211_ssid_eq - compare two SSIDs + * @a: first SSID + * @b: second SSID + * + * Return: %true if SSIDs are equal, %false otherwise. + */ +static inline bool +cfg80211_ssid_eq(struct cfg80211_ssid *a, struct cfg80211_ssid *b) +{ + if (WARN_ON(!a || !b)) + return false; + if (a->ssid_len != b->ssid_len) + return false; + return memcmp(a->ssid, b->ssid, a->ssid_len) ? false : true; +} + +/** * cfg80211_inform_bss_data - inform cfg80211 of a new BSS * * @wiphy: the wiphy reporting the BSS @@ -7346,8 +7368,6 @@ void cfg80211_auth_timeout(struct net_device *dev, const u8 *addr); /** * struct cfg80211_rx_assoc_resp_data - association response data - * @bss: the BSS that association was requested with, ownership of the pointer - * moves to cfg80211 in the call to cfg80211_rx_assoc_resp() * @buf: (Re)Association Response frame (header + body) * @len: length of the frame data * @uapsd_queues: bitmap of queues configured for uapsd. Same format @@ -7357,6 +7377,8 @@ void cfg80211_auth_timeout(struct net_device *dev, const u8 *addr); * @ap_mld_addr: AP MLD address (in case of MLO) * @links: per-link information indexed by link ID, use links[0] for * non-MLO connections + * @links.bss: the BSS that association was requested with, ownership of the + * pointer moves to cfg80211 in the call to cfg80211_rx_assoc_resp() * @links.status: Set this (along with a BSS pointer) for links that * were rejected by the AP. */ @@ -9375,6 +9397,16 @@ bool cfg80211_valid_disable_subchannel_bitmap(u16 *bitmap, */ void cfg80211_links_removed(struct net_device *dev, u16 link_mask); +/** + * cfg80211_schedule_channels_check - schedule regulatory check if needed + * @wdev: the wireless device to check + * + * In case the device supports NO_IR or DFS relaxations, schedule regulatory + * channels check, as previous concurrent operation conditions may not + * hold anymore. + */ +void cfg80211_schedule_channels_check(struct wireless_dev *wdev); + #ifdef CONFIG_CFG80211_DEBUGFS /** * wiphy_locked_debugfs_read - do a locked read in debugfs diff --git a/include/net/genetlink.h b/include/net/genetlink.h index 85c63d4f16dd..e61469129402 100644 --- a/include/net/genetlink.h +++ b/include/net/genetlink.h @@ -8,16 +8,19 @@ #define GENLMSG_DEFAULT_SIZE (NLMSG_DEFAULT_SIZE - GENL_HDRLEN) +/* Binding to multicast group requires %CAP_NET_ADMIN */ +#define GENL_MCAST_CAP_NET_ADMIN BIT(0) +/* Binding to multicast group requires %CAP_SYS_ADMIN */ +#define GENL_MCAST_CAP_SYS_ADMIN BIT(1) + /** * struct genl_multicast_group - generic netlink multicast group * @name: name of the multicast group, names are per-family - * @flags: GENL_* flags (%GENL_ADMIN_PERM or %GENL_UNS_ADMIN_PERM) - * @cap_sys_admin: whether %CAP_SYS_ADMIN is required for binding + * @flags: GENL_MCAST_* flags */ struct genl_multicast_group { char name[GENL_NAMSIZ]; u8 flags; - u8 cap_sys_admin:1; }; struct genl_split_ops; diff --git a/include/net/inet_hashtables.h b/include/net/inet_hashtables.h index 3ecfeadbfa06..7f1b38458743 100644 --- a/include/net/inet_hashtables.h +++ b/include/net/inet_hashtables.h @@ -88,7 +88,7 @@ struct inet_bind_bucket { unsigned short fast_sk_family; bool fast_ipv6_only; struct hlist_node node; - struct hlist_head owners; + struct hlist_head bhash2; }; struct inet_bind2_bucket { @@ -96,22 +96,17 @@ struct inet_bind2_bucket { int l3mdev; unsigned short port; #if IS_ENABLED(CONFIG_IPV6) - unsigned short family; -#endif - union { -#if IS_ENABLED(CONFIG_IPV6) - struct in6_addr v6_rcv_saddr; + unsigned short addr_type; + struct in6_addr v6_rcv_saddr; +#define rcv_saddr v6_rcv_saddr.s6_addr32[3] +#else + __be32 rcv_saddr; #endif - __be32 rcv_saddr; - }; /* Node in the bhash2 inet_bind_hashbucket chain */ struct hlist_node node; + struct hlist_node bhash_node; /* List of sockets hashed to this bucket */ struct hlist_head owners; - /* bhash has twsk in owners, but bhash2 has twsk in - * deathrow not to add a member in struct sock_common. - */ - struct hlist_head deathrow; }; static inline struct net *ib_net(const struct inet_bind_bucket *ib) @@ -241,7 +236,7 @@ bool inet_bind_bucket_match(const struct inet_bind_bucket *tb, struct inet_bind2_bucket * inet_bind2_bucket_create(struct kmem_cache *cachep, struct net *net, struct inet_bind_hashbucket *head, - unsigned short port, int l3mdev, + struct inet_bind_bucket *tb, const struct sock *sk); void inet_bind2_bucket_destroy(struct kmem_cache *cachep, diff --git a/include/net/inet_timewait_sock.h b/include/net/inet_timewait_sock.h index b14999ff55db..f28da08a37b4 100644 --- a/include/net/inet_timewait_sock.h +++ b/include/net/inet_timewait_sock.h @@ -75,13 +75,9 @@ struct inet_timewait_sock { struct timer_list tw_timer; struct inet_bind_bucket *tw_tb; struct inet_bind2_bucket *tw_tb2; - struct hlist_node tw_bind2_node; }; #define tw_tclass tw_tos -#define twsk_for_each_bound_bhash2(__tw, list) \ - hlist_for_each_entry(__tw, list, tw_bind2_node) - static inline struct inet_timewait_sock *inet_twsk(const struct sock *sk) { return (struct inet_timewait_sock *)sk; diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 78d38dd88aba..cf25ea21d770 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -784,11 +784,6 @@ static inline bool ipv6_addr_v4mapped(const struct in6_addr *a) cpu_to_be32(0x0000ffff))) == 0UL; } -static inline bool ipv6_addr_v4mapped_any(const struct in6_addr *a) -{ - return ipv6_addr_v4mapped(a) && ipv4_is_zeronet(a->s6_addr32[3]); -} - static inline bool ipv6_addr_v4mapped_loopback(const struct in6_addr *a) { return ipv6_addr_v4mapped(a) && ipv4_is_loopback(a->s6_addr32[3]); diff --git a/include/net/iucv/iucv.h b/include/net/iucv/iucv.h index f9e88401d7da..8b2055d64a6b 100644 --- a/include/net/iucv/iucv.h +++ b/include/net/iucv/iucv.h @@ -80,7 +80,7 @@ struct iucv_array { u32 length; } __attribute__ ((aligned (8))); -extern struct bus_type iucv_bus; +extern const struct bus_type iucv_bus; extern struct device *iucv_root; /* @@ -489,7 +489,7 @@ struct iucv_interface { int (*path_sever)(struct iucv_path *path, u8 userdata[16]); int (*iucv_register)(struct iucv_handler *handler, int smp); void (*iucv_unregister)(struct iucv_handler *handler, int smp); - struct bus_type *bus; + const struct bus_type *bus; struct device *root; }; diff --git a/include/net/mac80211.h b/include/net/mac80211.h index 77a71b1396b1..d400fe2e8668 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -476,9 +476,9 @@ struct ieee80211_ba_event { /** * struct ieee80211_event - event to be sent to the driver * @type: The event itself. See &enum ieee80211_event_type. - * @rssi: relevant if &type is %RSSI_EVENT - * @mlme: relevant if &type is %AUTH_EVENT - * @ba: relevant if &type is %BAR_RX_EVENT or %BA_FRAME_TIMEOUT + * @u.rssi: relevant if &type is %RSSI_EVENT + * @u.mlme: relevant if &type is %AUTH_EVENT + * @u.ba: relevant if &type is %BAR_RX_EVENT or %BA_FRAME_TIMEOUT * @u:union holding the fields above */ struct ieee80211_event { @@ -541,8 +541,6 @@ struct ieee80211_fils_discovery { * @link_id: link ID, or 0 for non-MLO * @htc_trig_based_pkt_ext: default PE in 4us units, if BSS supports HE * @uora_exists: is the UORA element advertised by AP - * @ack_enabled: indicates support to receive a multi-TID that solicits either - * ACK, BACK or both * @uora_ocw_range: UORA element's OCW Range field * @frame_time_rts_th: HE duration RTS threshold, in units of 32us * @he_support: does this BSS support HE @@ -1150,11 +1148,6 @@ ieee80211_rate_get_vht_nss(const struct ieee80211_tx_rate *rate) * @ack: union part for pure ACK data * @ack.cookie: cookie for the ACK * @driver_data: array of driver_data pointers - * @ampdu_ack_len: number of acked aggregated frames. - * relevant only if IEEE80211_TX_STAT_AMPDU was set. - * @ampdu_len: number of aggregated frames. - * relevant only if IEEE80211_TX_STAT_AMPDU was set. - * @ack_signal: signal strength of the ACK frame */ struct ieee80211_tx_info { /* common information */ @@ -1362,6 +1355,9 @@ ieee80211_tx_info_clear_status(struct ieee80211_tx_info *info) * the frame. * @RX_FLAG_FAILED_PLCP_CRC: Set this flag if the PCLP check failed on * the frame. + * @RX_FLAG_MACTIME: The timestamp passed in the RX status (@mactime + * field) is valid if this field is non-zero, and the position + * where the timestamp was sampled depends on the value. * @RX_FLAG_MACTIME_START: The timestamp passed in the RX status (@mactime * field) is valid and contains the time the first symbol of the MPDU * was received. This is useful in monitor mode and for proper IBSS @@ -1371,6 +1367,11 @@ ieee80211_tx_info_clear_status(struct ieee80211_tx_info *info) * (including FCS) was received. * @RX_FLAG_MACTIME_PLCP_START: The timestamp passed in the RX status (@mactime * field) is valid and contains the time the SYNC preamble was received. + * @RX_FLAG_MACTIME_IS_RTAP_TS64: The timestamp passed in the RX status @mactime + * is only for use in the radiotap timestamp header, not otherwise a valid + * @mactime value. Note this is a separate flag so that we continue to see + * %RX_FLAG_MACTIME as unset. Also note that in this case the timestamp is + * reported to be 64 bits wide, not just 32. * @RX_FLAG_NO_SIGNAL_VAL: The signal strength value is not present. * Valid only for data frames (mainly A-MPDU) * @RX_FLAG_AMPDU_DETAILS: A-MPDU details are known, in particular the reference @@ -1441,12 +1442,12 @@ ieee80211_tx_info_clear_status(struct ieee80211_tx_info *info) enum mac80211_rx_flags { RX_FLAG_MMIC_ERROR = BIT(0), RX_FLAG_DECRYPTED = BIT(1), - RX_FLAG_MACTIME_PLCP_START = BIT(2), + RX_FLAG_ONLY_MONITOR = BIT(2), RX_FLAG_MMIC_STRIPPED = BIT(3), RX_FLAG_IV_STRIPPED = BIT(4), RX_FLAG_FAILED_FCS_CRC = BIT(5), RX_FLAG_FAILED_PLCP_CRC = BIT(6), - RX_FLAG_MACTIME_START = BIT(7), + RX_FLAG_MACTIME_IS_RTAP_TS64 = BIT(7), RX_FLAG_NO_SIGNAL_VAL = BIT(8), RX_FLAG_AMPDU_DETAILS = BIT(9), RX_FLAG_PN_VALIDATED = BIT(10), @@ -1455,8 +1456,10 @@ enum mac80211_rx_flags { RX_FLAG_AMPDU_IS_LAST = BIT(13), RX_FLAG_AMPDU_DELIM_CRC_ERROR = BIT(14), RX_FLAG_AMPDU_DELIM_CRC_KNOWN = BIT(15), - RX_FLAG_MACTIME_END = BIT(16), - RX_FLAG_ONLY_MONITOR = BIT(17), + RX_FLAG_MACTIME = BIT(16) | BIT(17), + RX_FLAG_MACTIME_PLCP_START = 1 << 16, + RX_FLAG_MACTIME_START = 2 << 16, + RX_FLAG_MACTIME_END = 3 << 16, RX_FLAG_SKIP_MONITOR = BIT(18), RX_FLAG_AMSDU_MORE = BIT(19), RX_FLAG_RADIOTAP_TLV_AT_END = BIT(20), @@ -2835,8 +2838,6 @@ enum ieee80211_hw_flags { * the default is _GI | _BANDWIDTH. * Use the %IEEE80211_RADIOTAP_VHT_KNOWN_\* values. * - * @radiotap_he: HE radiotap validity flags - * * @radiotap_timestamp: Information for the radiotap timestamp field; if the * @units_pos member is set to a non-negative value then the timestamp * field will be added and populated from the &struct ieee80211_rx_status @@ -4271,6 +4272,8 @@ struct ieee80211_prep_tx_info { * disable background CAC/radar detection. * @net_fill_forward_path: Called from .ndo_fill_forward_path in order to * resolve a path for hardware flow offloading + * @can_activate_links: Checks if a specific active_links bitmap is + * supported by the driver. * @change_vif_links: Change the valid links on an interface, note that while * removing the old link information is still valid (link_conf pointer), * but may immediately disappear after the function returns. The old or @@ -4651,6 +4654,9 @@ struct ieee80211_ops { struct ieee80211_sta *sta, struct net_device_path_ctx *ctx, struct net_device_path *path); + bool (*can_activate_links)(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + u16 active_links); int (*change_vif_links)(struct ieee80211_hw *hw, struct ieee80211_vif *vif, u16 old_links, u16 new_links, diff --git a/include/net/macsec.h b/include/net/macsec.h index ebf9bc54036a..dbd22180cc5c 100644 --- a/include/net/macsec.h +++ b/include/net/macsec.h @@ -247,6 +247,23 @@ struct macsec_secy { /** * struct macsec_context - MACsec context for hardware offloading + * @netdev: a valid pointer to a struct net_device if @offload == + * MACSEC_OFFLOAD_MAC + * @phydev: a valid pointer to a struct phy_device if @offload == + * MACSEC_OFFLOAD_PHY + * @offload: MACsec offload status + * @secy: pointer to a MACsec SecY + * @rx_sc: pointer to a RX SC + * @update_pn: when updating the SA, update the next PN + * @assoc_num: association number of the target SA + * @key: key of the target SA + * @rx_sa: pointer to an RX SA if a RX SA is added/updated/removed + * @tx_sa: pointer to an TX SA if a TX SA is added/updated/removed + * @tx_sc_stats: pointer to TX SC stats structure + * @tx_sa_stats: pointer to TX SA stats structure + * @rx_sc_stats: pointer to RX SC stats structure + * @rx_sa_stats: pointer to RX SA stats structure + * @dev_stats: pointer to dev stats structure */ struct macsec_context { union { @@ -277,6 +294,33 @@ struct macsec_context { /** * struct macsec_ops - MACsec offloading operations + * @mdo_dev_open: called when the MACsec interface transitions to the up state + * @mdo_dev_stop: called when the MACsec interface transitions to the down + * state + * @mdo_add_secy: called when a new SecY is added + * @mdo_upd_secy: called when the SecY flags are changed or the MAC address of + * the MACsec interface is changed + * @mdo_del_secy: called when the hw offload is disabled or the MACsec + * interface is removed + * @mdo_add_rxsc: called when a new RX SC is added + * @mdo_upd_rxsc: called when a certain RX SC is updated + * @mdo_del_rxsc: called when a certain RX SC is removed + * @mdo_add_rxsa: called when a new RX SA is added + * @mdo_upd_rxsa: called when a certain RX SA is updated + * @mdo_del_rxsa: called when a certain RX SA is removed + * @mdo_add_txsa: called when a new TX SA is added + * @mdo_upd_txsa: called when a certain TX SA is updated + * @mdo_del_txsa: called when a certain TX SA is removed + * @mdo_get_dev_stats: called when dev stats are read + * @mdo_get_tx_sc_stats: called when TX SC stats are read + * @mdo_get_tx_sa_stats: called when TX SA stats are read + * @mdo_get_rx_sc_stats: called when RX SC stats are read + * @mdo_get_rx_sa_stats: called when RX SA stats are read + * @mdo_insert_tx_tag: called to insert the TX tag + * @needed_headroom: number of bytes reserved at the beginning of the sk_buff + * for the TX tag + * @needed_tailroom: number of bytes reserved at the end of the sk_buff for the + * TX tag */ struct macsec_ops { /* Device wide */ @@ -303,6 +347,11 @@ struct macsec_ops { int (*mdo_get_tx_sa_stats)(struct macsec_context *ctx); int (*mdo_get_rx_sc_stats)(struct macsec_context *ctx); int (*mdo_get_rx_sa_stats)(struct macsec_context *ctx); + /* Offload tag */ + int (*mdo_insert_tx_tag)(struct phy_device *phydev, + struct sk_buff *skb); + unsigned int needed_headroom; + unsigned int needed_tailroom; }; void macsec_pn_wrapped(struct macsec_secy *secy, struct macsec_tx_sa *tx_sa); @@ -325,4 +374,9 @@ static inline void *macsec_netdev_priv(const struct net_device *dev) return netdev_priv(dev); } +static inline u64 sci_to_cpu(sci_t sci) +{ + return be64_to_cpu((__force __be64)sci); +} + #endif /* _NET_MACSEC_H_ */ diff --git a/include/net/netfilter/nf_flow_table.h b/include/net/netfilter/nf_flow_table.h index 692d5955911c..956c752ceb31 100644 --- a/include/net/netfilter/nf_flow_table.h +++ b/include/net/netfilter/nf_flow_table.h @@ -74,12 +74,13 @@ enum nf_flowtable_flags { }; struct nf_flowtable { - struct list_head list; - struct rhashtable rhashtable; - int priority; + unsigned int flags; /* readonly in datapath */ + int priority; /* control path (padding hole) */ + struct rhashtable rhashtable; /* datapath, read-mostly members come first */ + + struct list_head list; /* slowpath parts */ const struct nf_flowtable_type *type; struct delayed_work gc_work; - unsigned int flags; struct flow_block flow_block; struct rw_semaphore flow_block_lock; /* Guards flow_block */ possible_net_t net; diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 1d70c2c1572f..ba3e1b315de8 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -19,6 +19,7 @@ #include <net/gen_stats.h> #include <net/rtnetlink.h> #include <net/flow_offload.h> +#include <linux/xarray.h> struct Qdisc_ops; struct qdisc_walker; @@ -456,6 +457,7 @@ struct tcf_chain { }; struct tcf_block { + struct xarray ports; /* datapath accessible */ /* Lock protects tcf_block and lifetime-management data of chains * attached to the block (refcnt, action_refcnt, explicitly_created). */ @@ -482,6 +484,8 @@ struct tcf_block { struct mutex proto_destroy_lock; /* Lock for proto_destroy hashtable. */ }; +struct tcf_block *tcf_block_lookup(struct net *net, u32 block_index); + static inline bool lockdep_tcf_chain_is_locked(struct tcf_chain *chain) { return lockdep_is_held(&chain->filter_chain_lock); diff --git a/include/net/smc.h b/include/net/smc.h index a002552be29c..c9dcb30e3fd9 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -52,9 +52,14 @@ struct smcd_dmb { struct smcd_dev; struct ism_client; +struct smcd_gid { + u64 gid; + u64 gid_ext; +}; + struct smcd_ops { - int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid, - u32 vid); + int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 vid_valid, u32 vid); int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, struct ism_client *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); @@ -62,14 +67,13 @@ struct smcd_ops { int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct smcd_dev *dev); int (*reset_vlan_required)(struct smcd_dev *dev); - int (*signal_event)(struct smcd_dev *dev, u64 rgid, u32 trigger_irq, - u32 event_code, u64 info); + int (*signal_event)(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info); int (*move_data)(struct smcd_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); int (*supports_v2)(void); - u8* (*get_system_eid)(void); - u64 (*get_local_gid)(struct smcd_dev *dev); + void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); u16 (*get_chid)(struct smcd_dev *dev); struct device* (*get_dev)(struct smcd_dev *dev); }; diff --git a/include/net/sock.h b/include/net/sock.h index ba6642811db4..a7f815c7cfdf 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -76,19 +76,6 @@ * the other protocols. */ -/* Define this to get the SOCK_DBG debugging facility. */ -#define SOCK_DEBUGGING -#ifdef SOCK_DEBUGGING -#define SOCK_DEBUG(sk, msg...) do { if ((sk) && sock_flag((sk), SOCK_DBG)) \ - printk(KERN_DEBUG msg); } while (0) -#else -/* Validate arguments and do nothing */ -static inline __printf(2, 3) -void SOCK_DEBUG(const struct sock *sk, const char *msg, ...) -{ -} -#endif - /* This is the per-socket lock. The spinlock provides a synchronization * between user contexts and software interrupt processing, whereas the * mini-semaphore synchronizes multiple users amongst themselves. @@ -277,8 +264,6 @@ struct sk_filter; * @sk_pacing_status: Pacing status (requested, handled by sch_fq) * @sk_max_pacing_rate: Maximum pacing rate (%SO_MAX_PACING_RATE) * @sk_sndbuf: size of send buffer in bytes - * @__sk_flags_offset: empty field used to determine location of bitfield - * @sk_padding: unused element for alignment * @sk_no_check_tx: %SO_NO_CHECK setting, set checksum in TX packets * @sk_no_check_rx: allow zero checksum in RX packets * @sk_route_caps: route capabilities (e.g. %NETIF_F_TSO) @@ -352,7 +337,6 @@ struct sk_filter; * @sk_txtime_report_errors: set report errors mode for SO_TXTIME * @sk_txtime_unused: unused txtime flags * @ns_tracker: tracker for netns reference - * @sk_bind2_node: bind node in the bhash2 table */ struct sock { /* @@ -544,7 +528,6 @@ struct sock { #endif struct rcu_head sk_rcu; netns_tracker ns_tracker; - struct hlist_node sk_bind2_node; }; enum sk_pacing { @@ -873,16 +856,6 @@ static inline void sk_add_bind_node(struct sock *sk, hlist_add_head(&sk->sk_bind_node, list); } -static inline void __sk_del_bind2_node(struct sock *sk) -{ - __hlist_del(&sk->sk_bind2_node); -} - -static inline void sk_add_bind2_node(struct sock *sk, struct hlist_head *list) -{ - hlist_add_head(&sk->sk_bind2_node, list); -} - #define sk_for_each(__sk, list) \ hlist_for_each_entry(__sk, list, sk_node) #define sk_for_each_rcu(__sk, list) \ @@ -900,8 +873,6 @@ static inline void sk_add_bind2_node(struct sock *sk, struct hlist_head *list) hlist_for_each_entry_safe(__sk, tmp, list, sk_node) #define sk_for_each_bound(__sk, list) \ hlist_for_each_entry(__sk, list, sk_bind_node) -#define sk_for_each_bound_bhash2(__sk, list) \ - hlist_for_each_entry(__sk, list, sk_bind2_node) /** * sk_for_each_entry_offset_rcu - iterate over a list at a given struct offset diff --git a/include/net/tc_act/tc_ipt.h b/include/net/tc_act/tc_ipt.h deleted file mode 100644 index 4225fcb1c6ba..000000000000 --- a/include/net/tc_act/tc_ipt.h +++ /dev/null @@ -1,17 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __NET_TC_IPT_H -#define __NET_TC_IPT_H - -#include <net/act_api.h> - -struct xt_entry_target; - -struct tcf_ipt { - struct tc_action common; - u32 tcfi_hook; - char *tcfi_tname; - struct xt_entry_target *tcfi_t; -}; -#define to_ipt(a) ((struct tcf_ipt *)a) - -#endif /* __NET_TC_IPT_H */ diff --git a/include/net/tc_act/tc_mirred.h b/include/net/tc_act/tc_mirred.h index 32ce8ea36950..75722d967bf2 100644 --- a/include/net/tc_act/tc_mirred.h +++ b/include/net/tc_act/tc_mirred.h @@ -8,6 +8,7 @@ struct tcf_mirred { struct tc_action common; int tcfm_eaction; + u32 tcfm_blockid; bool tcfm_mac_header_xmit; struct net_device __rcu *tcfm_dev; netdevice_tracker tcfm_dev_tracker; diff --git a/include/net/tc_wrapper.h b/include/net/tc_wrapper.h index a6d481b5bcbc..a608546bcefc 100644 --- a/include/net/tc_wrapper.h +++ b/include/net/tc_wrapper.h @@ -117,10 +117,6 @@ static inline int tc_act(struct sk_buff *skb, const struct tc_action *a, if (a->ops->act == tcf_ife_act) return tcf_ife_act(skb, a, res); #endif -#if IS_BUILTIN(CONFIG_NET_ACT_IPT) - if (a->ops->act == tcf_ipt_act) - return tcf_ipt_act(skb, a, res); -#endif #if IS_BUILTIN(CONFIG_NET_ACT_SIMP) if (a->ops->act == tcf_simp_act) return tcf_simp_act(skb, a, res); diff --git a/include/uapi/linux/ethtool.h b/include/uapi/linux/ethtool.h index 0787d561ace0..60801df9d8c0 100644 --- a/include/uapi/linux/ethtool.h +++ b/include/uapi/linux/ethtool.h @@ -2139,18 +2139,6 @@ enum ethtool_reset_flags { * refused. For drivers: ignore this field (use kernel's * __ETHTOOL_LINK_MODE_MASK_NBITS instead), any change to it will * be overwritten by kernel. - * @supported: Bitmap with each bit meaning given by - * %ethtool_link_mode_bit_indices for the link modes, physical - * connectors and other link features for which the interface - * supports autonegotiation or auto-detection. Read-only. - * @advertising: Bitmap with each bit meaning given by - * %ethtool_link_mode_bit_indices for the link modes, physical - * connectors and other link features that are advertised through - * autonegotiation or enabled for auto-detection. - * @lp_advertising: Bitmap with each bit meaning given by - * %ethtool_link_mode_bit_indices for the link modes, and other - * link features that the link partner advertised through - * autonegotiation; 0 if unknown or not applicable. Read-only. * @transceiver: Used to distinguish different possible PHY types, * reported consistently by PHYLIB. Read-only. * @master_slave_cfg: Master/slave port mode. @@ -2192,6 +2180,21 @@ enum ethtool_reset_flags { * %set_link_ksettings() should validate all fields other than @cmd * and @link_mode_masks_nwords that are not described as read-only or * deprecated, and must ignore all fields described as read-only. + * + * @link_mode_masks is divided into three bitfields, each of length + * @link_mode_masks_nwords: + * - supported: Bitmap with each bit meaning given by + * %ethtool_link_mode_bit_indices for the link modes, physical + * connectors and other link features for which the interface + * supports autonegotiation or auto-detection. Read-only. + * - advertising: Bitmap with each bit meaning given by + * %ethtool_link_mode_bit_indices for the link modes, physical + * connectors and other link features that are advertised through + * autonegotiation or enabled for auto-detection. + * - lp_advertising: Bitmap with each bit meaning given by + * %ethtool_link_mode_bit_indices for the link modes, and other + * link features that the link partner advertised through + * autonegotiation; 0 if unknown or not applicable. Read-only. */ struct ethtool_link_settings { __u32 cmd; @@ -2216,4 +2219,20 @@ struct ethtool_link_settings { * __u32 map_lp_advertising[link_mode_masks_nwords]; */ }; + +/** + * enum phy_upstream - Represents the upstream component a given PHY device + * is connected to, as in what is on the other end of the MII bus. Most PHYs + * will be attached to an Ethernet MAC controller, but in some cases, there's + * an intermediate PHY used as a media-converter, which will driver another + * MII interface as its output. + * @PHY_UPSTREAM_MAC: Upstream component is a MAC (a switch port, + * or ethernet controller) + * @PHY_UPSTREAM_PHY: Upstream component is a PHY (likely a media converter) + */ +enum phy_upstream { + PHY_UPSTREAM_MAC, + PHY_UPSTREAM_PHY, +}; + #endif /* _UAPI_LINUX_ETHTOOL_H */ diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h index 3f89074aa06c..00cd7ad16709 100644 --- a/include/uapi/linux/ethtool_netlink.h +++ b/include/uapi/linux/ethtool_netlink.h @@ -57,6 +57,7 @@ enum { ETHTOOL_MSG_PLCA_GET_STATUS, ETHTOOL_MSG_MM_GET, ETHTOOL_MSG_MM_SET, + ETHTOOL_MSG_PHY_GET, /* add new constants above here */ __ETHTOOL_MSG_USER_CNT, @@ -109,6 +110,8 @@ enum { ETHTOOL_MSG_PLCA_NTF, ETHTOOL_MSG_MM_GET_REPLY, ETHTOOL_MSG_MM_NTF, + ETHTOOL_MSG_PHY_GET_REPLY, + ETHTOOL_MSG_PHY_NTF, /* add new constants above here */ __ETHTOOL_MSG_KERNEL_CNT, @@ -133,6 +136,7 @@ enum { ETHTOOL_A_HEADER_DEV_INDEX, /* u32 */ ETHTOOL_A_HEADER_DEV_NAME, /* string */ ETHTOOL_A_HEADER_FLAGS, /* u32 - ETHTOOL_FLAG_* */ + ETHTOOL_A_HEADER_PHY_INDEX, /* u32 */ /* add new constants above here */ __ETHTOOL_A_HEADER_CNT, @@ -976,6 +980,32 @@ enum { ETHTOOL_A_MM_MAX = (__ETHTOOL_A_MM_CNT - 1) }; +enum { + ETHTOOL_A_PHY_UPSTREAM_UNSPEC, + ETHTOOL_A_PHY_UPSTREAM_INDEX, /* u32 */ + ETHTOOL_A_PHY_UPSTREAM_SFP_NAME, /* string */ + + /* add new constants above here */ + __ETHTOOL_A_PHY_UPSTREAM_CNT, + ETHTOOL_A_PHY_UPSTREAM_MAX = (__ETHTOOL_A_PHY_UPSTREAM_CNT - 1) +}; + +enum { + ETHTOOL_A_PHY_UNSPEC, + ETHTOOL_A_PHY_HEADER, /* nest - _A_HEADER_* */ + ETHTOOL_A_PHY_INDEX, /* u32 */ + ETHTOOL_A_PHY_DRVNAME, /* string */ + ETHTOOL_A_PHY_NAME, /* string */ + ETHTOOL_A_PHY_UPSTREAM_TYPE, /* u8 */ + ETHTOOL_A_PHY_UPSTREAM, /* nest - _A_PHY_UPSTREAM_* */ + ETHTOOL_A_PHY_DOWNSTREAM_SFP_NAME, /* string */ + ETHTOOL_A_PHY_ID, /* u32 */ + + /* add new constants above here */ + __ETHTOOL_A_PHY_CNT, + ETHTOOL_A_PHY_MAX = (__ETHTOOL_A_PHY_CNT - 1) +}; + /* generic netlink info */ #define ETHTOOL_GENL_NAME "ethtool" #define ETHTOOL_GENL_VERSION 1 diff --git a/include/uapi/linux/nl80211.h b/include/uapi/linux/nl80211.h index a682b54bd3ba..1ccdcae24372 100644 --- a/include/uapi/linux/nl80211.h +++ b/include/uapi/linux/nl80211.h @@ -4256,6 +4256,14 @@ enum nl80211_wmm_rule { * in current regulatory domain. * @NL80211_FREQUENCY_ATTR_PSD: Power spectral density (in dBm) that * is allowed on this channel in current regulatory domain. + * @NL80211_FREQUENCY_ATTR_DFS_CONCURRENT: Operation on this channel is + * allowed for peer-to-peer or adhoc communication under the control + * of a DFS master which operates on the same channel (FCC-594280 D01 + * Section B.3). Should be used together with %NL80211_RRF_DFS only. + * @NL80211_FREQUENCY_ATTR_NO_UHB_VLP_CLIENT: Client connection to VLP AP + * not allowed using this channel + * @NL80211_FREQUENCY_ATTR_NO_UHB_AFC_CLIENT: Client connection to AFC AP + * not allowed using this channel * @NL80211_FREQUENCY_ATTR_MAX: highest frequency attribute number * currently defined * @__NL80211_FREQUENCY_ATTR_AFTER_LAST: internal use @@ -4295,6 +4303,9 @@ enum nl80211_frequency_attr { NL80211_FREQUENCY_ATTR_NO_320MHZ, NL80211_FREQUENCY_ATTR_NO_EHT, NL80211_FREQUENCY_ATTR_PSD, + NL80211_FREQUENCY_ATTR_DFS_CONCURRENT, + NL80211_FREQUENCY_ATTR_NO_UHB_VLP_CLIENT, + NL80211_FREQUENCY_ATTR_NO_UHB_AFC_CLIENT, /* keep last */ __NL80211_FREQUENCY_ATTR_AFTER_LAST, @@ -4500,6 +4511,12 @@ enum nl80211_sched_scan_match_attr { * @NL80211_RRF_NO_320MHZ: 320MHz operation not allowed * @NL80211_RRF_NO_EHT: EHT operation not allowed * @NL80211_RRF_PSD: Ruleset has power spectral density value + * @NL80211_RRF_DFS_CONCURRENT: Operation on this channel is allowed for + peer-to-peer or adhoc communication under the control of a DFS master + which operates on the same channel (FCC-594280 D01 Section B.3). + Should be used together with %NL80211_RRF_DFS only. + * @NL80211_RRF_NO_UHB_VLP_CLIENT: Client connection to VLP AP not allowed + * @NL80211_RRF_NO_UHB_AFC_CLIENT: Client connection to AFC AP not allowed */ enum nl80211_reg_rule_flags { NL80211_RRF_NO_OFDM = 1<<0, @@ -4521,6 +4538,9 @@ enum nl80211_reg_rule_flags { NL80211_RRF_NO_320MHZ = 1<<18, NL80211_RRF_NO_EHT = 1<<19, NL80211_RRF_PSD = 1<<20, + NL80211_RRF_DFS_CONCURRENT = 1<<21, + NL80211_RRF_NO_UHB_VLP_CLIENT = 1<<22, + NL80211_RRF_NO_UHB_AFC_CLIENT = 1<<23, }; #define NL80211_RRF_PASSIVE_SCAN NL80211_RRF_NO_IR @@ -5076,9 +5096,12 @@ enum nl80211_bss_use_for { * BSS isn't possible * @NL80211_BSS_CANNOT_USE_NSTR_NONPRIMARY: NSTR nonprimary links aren't * supported by the device, and this BSS entry represents one. + * @NL80211_BSS_CANNOT_USE_UHB_PWR_MISMATCH: STA is not supporting + * the AP power type (SP, VLP, AP) that the AP uses. */ enum nl80211_bss_cannot_use_reasons { NL80211_BSS_CANNOT_USE_NSTR_NONPRIMARY = 1 << 0, + NL80211_BSS_CANNOT_USE_UHB_PWR_MISMATCH = 1 << 1, }; /** @@ -6492,6 +6515,11 @@ enum nl80211_feature_flags { * @NL80211_EXT_FEATURE_OWE_OFFLOAD_AP: Driver/Device wants to do OWE DH IE * handling in AP mode. * + * @NL80211_EXT_FEATURE_DFS_CONCURRENT: The device supports peer-to-peer or + * ad hoc operation on DFS channels under the control of a concurrent + * DFS master on the same channel as described in FCC-594280 D01 + * (Section B.3). This, for example, allows P2P GO and P2P clients to + * operate on DFS channels as long as there's a concurrent BSS connection. * @NUM_NL80211_EXT_FEATURES: number of extended features. * @MAX_NL80211_EXT_FEATURES: highest extended feature index. */ @@ -6565,6 +6593,7 @@ enum nl80211_ext_feature_index { NL80211_EXT_FEATURE_AUTH_AND_DEAUTH_RANDOM_TA, NL80211_EXT_FEATURE_OWE_OFFLOAD, NL80211_EXT_FEATURE_OWE_OFFLOAD_AP, + NL80211_EXT_FEATURE_DFS_CONCURRENT, /* add new features before the definition below */ NUM_NL80211_EXT_FEATURES, diff --git a/include/uapi/linux/pkt_cls.h b/include/uapi/linux/pkt_cls.h index c7082cc60d21..2fec9b51d28d 100644 --- a/include/uapi/linux/pkt_cls.h +++ b/include/uapi/linux/pkt_cls.h @@ -99,7 +99,7 @@ enum { * versions. */ #define TCA_ACT_GACT 5 -#define TCA_ACT_IPT 6 +#define TCA_ACT_IPT 6 /* obsoleted, can be reused */ #define TCA_ACT_PEDIT 7 #define TCA_ACT_MIRRED 8 #define TCA_ACT_NAT 9 @@ -120,7 +120,7 @@ enum tca_id { TCA_ID_UNSPEC = 0, TCA_ID_POLICE = 1, TCA_ID_GACT = TCA_ACT_GACT, - TCA_ID_IPT = TCA_ACT_IPT, + TCA_ID_IPT = TCA_ACT_IPT, /* Obsoleted, can be reused */ TCA_ID_PEDIT = TCA_ACT_PEDIT, TCA_ID_MIRRED = TCA_ACT_MIRRED, TCA_ID_NAT = TCA_ACT_NAT, diff --git a/include/uapi/linux/smc.h b/include/uapi/linux/smc.h index 837fcd4b0abc..b531e3ef011a 100644 --- a/include/uapi/linux/smc.h +++ b/include/uapi/linux/smc.h @@ -160,6 +160,8 @@ enum { SMC_NLA_LGR_D_CHID, /* u16 */ SMC_NLA_LGR_D_PAD, /* flag */ SMC_NLA_LGR_D_V2_COMMON, /* nest */ + SMC_NLA_LGR_D_EXT_GID, /* u64 */ + SMC_NLA_LGR_D_PEER_EXT_GID, /* u64 */ __SMC_NLA_LGR_D_MAX, SMC_NLA_LGR_D_MAX = __SMC_NLA_LGR_D_MAX - 1 }; diff --git a/include/uapi/linux/smc_diag.h b/include/uapi/linux/smc_diag.h index 8cb3a6fef553..58eceb7f5df2 100644 --- a/include/uapi/linux/smc_diag.h +++ b/include/uapi/linux/smc_diag.h @@ -107,6 +107,8 @@ struct smcd_diag_dmbinfo { /* SMC-D Socket internals */ __aligned_u64 my_gid; /* My GID */ __aligned_u64 token; /* Token of DMB */ __aligned_u64 peer_token; /* Token of remote DMBE */ + __aligned_u64 peer_gid_ext; /* Peer GID (extended part) */ + __aligned_u64 my_gid_ext; /* My GID (extended part) */ }; #endif /* _UAPI_SMC_DIAG_H_ */ diff --git a/include/uapi/linux/tc_act/tc_ipt.h b/include/uapi/linux/tc_act/tc_ipt.h deleted file mode 100644 index c48d7da6750d..000000000000 --- a/include/uapi/linux/tc_act/tc_ipt.h +++ /dev/null @@ -1,20 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ -#ifndef __LINUX_TC_IPT_H -#define __LINUX_TC_IPT_H - -#include <linux/pkt_cls.h> - -enum { - TCA_IPT_UNSPEC, - TCA_IPT_TABLE, - TCA_IPT_HOOK, - TCA_IPT_INDEX, - TCA_IPT_CNT, - TCA_IPT_TM, - TCA_IPT_TARG, - TCA_IPT_PAD, - __TCA_IPT_MAX -}; -#define TCA_IPT_MAX (__TCA_IPT_MAX - 1) - -#endif diff --git a/include/uapi/linux/tc_act/tc_mirred.h b/include/uapi/linux/tc_act/tc_mirred.h index 2500a0005d05..c61e76f3c23b 100644 --- a/include/uapi/linux/tc_act/tc_mirred.h +++ b/include/uapi/linux/tc_act/tc_mirred.h @@ -21,6 +21,7 @@ enum { TCA_MIRRED_TM, TCA_MIRRED_PARMS, TCA_MIRRED_PAD, + TCA_MIRRED_BLOCKID, __TCA_MIRRED_MAX }; #define TCA_MIRRED_MAX (__TCA_MIRRED_MAX - 1) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index d56433bf8aba..51e8b4bee0c8 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6765,222 +6765,64 @@ int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *pr return btf_check_func_type_match(log, btf1, t1, btf2, t2); } -static int btf_check_func_arg_match(struct bpf_verifier_env *env, - const struct btf *btf, u32 func_id, - struct bpf_reg_state *regs, - bool ptr_to_mem_ok, - bool processing_call) +static bool btf_is_dynptr_ptr(const struct btf *btf, const struct btf_type *t) { - enum bpf_prog_type prog_type = resolve_prog_type(env->prog); - struct bpf_verifier_log *log = &env->log; - const char *func_name, *ref_tname; - const struct btf_type *t, *ref_t; - const struct btf_param *args; - u32 i, nargs, ref_id; - int ret; - - t = btf_type_by_id(btf, func_id); - if (!t || !btf_type_is_func(t)) { - /* These checks were already done by the verifier while loading - * struct bpf_func_info or in add_kfunc_call(). - */ - bpf_log(log, "BTF of func_id %u doesn't point to KIND_FUNC\n", - func_id); - return -EFAULT; - } - func_name = btf_name_by_offset(btf, t->name_off); - - t = btf_type_by_id(btf, t->type); - if (!t || !btf_type_is_func_proto(t)) { - bpf_log(log, "Invalid BTF of func %s\n", func_name); - return -EFAULT; - } - args = (const struct btf_param *)(t + 1); - nargs = btf_type_vlen(t); - if (nargs > MAX_BPF_FUNC_REG_ARGS) { - bpf_log(log, "Function %s has %d > %d args\n", func_name, nargs, - MAX_BPF_FUNC_REG_ARGS); - return -EINVAL; - } - - /* check that BTF function arguments match actual types that the - * verifier sees. - */ - for (i = 0; i < nargs; i++) { - enum bpf_arg_type arg_type = ARG_DONTCARE; - u32 regno = i + 1; - struct bpf_reg_state *reg = ®s[regno]; - - t = btf_type_skip_modifiers(btf, args[i].type, NULL); - if (btf_type_is_scalar(t)) { - if (reg->type == SCALAR_VALUE) - continue; - bpf_log(log, "R%d is not a scalar\n", regno); - return -EINVAL; - } - - if (!btf_type_is_ptr(t)) { - bpf_log(log, "Unrecognized arg#%d type %s\n", - i, btf_type_str(t)); - return -EINVAL; - } - - ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); - ref_tname = btf_name_by_offset(btf, ref_t->name_off); - - ret = check_func_arg_reg_off(env, reg, regno, arg_type); - if (ret < 0) - return ret; + const char *name; - if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { - /* If function expects ctx type in BTF check that caller - * is passing PTR_TO_CTX. - */ - if (reg->type != PTR_TO_CTX) { - bpf_log(log, - "arg#%d expected pointer to ctx, but got %s\n", - i, btf_type_str(t)); - return -EINVAL; - } - } else if (ptr_to_mem_ok && processing_call) { - const struct btf_type *resolve_ret; - u32 type_size; + t = btf_type_by_id(btf, t->type); /* skip PTR */ - resolve_ret = btf_resolve_size(btf, ref_t, &type_size); - if (IS_ERR(resolve_ret)) { - bpf_log(log, - "arg#%d reference type('%s %s') size cannot be determined: %ld\n", - i, btf_type_str(ref_t), ref_tname, - PTR_ERR(resolve_ret)); - return -EINVAL; - } + while (btf_type_is_modifier(t)) + t = btf_type_by_id(btf, t->type); - if (check_mem_reg(env, reg, regno, type_size)) - return -EINVAL; - } else { - bpf_log(log, "reg type unsupported for arg#%d function %s#%d\n", i, - func_name, func_id); - return -EINVAL; - } + /* allow either struct or struct forward declaration */ + if (btf_type_is_struct(t) || + (btf_type_is_fwd(t) && btf_type_kflag(t) == 0)) { + name = btf_str_by_offset(btf, t->name_off); + return name && strcmp(name, "bpf_dynptr") == 0; } - return 0; -} - -/* Compare BTF of a function declaration with given bpf_reg_state. - * Returns: - * EFAULT - there is a verifier bug. Abort verification. - * EINVAL - there is a type mismatch or BTF is not available. - * 0 - BTF matches with what bpf_reg_state expects. - * Only PTR_TO_CTX and SCALAR_VALUE states are recognized. - */ -int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *regs) -{ - struct bpf_prog *prog = env->prog; - struct btf *btf = prog->aux->btf; - bool is_global; - u32 btf_id; - int err; - - if (!prog->aux->func_info) - return -EINVAL; - - btf_id = prog->aux->func_info[subprog].type_id; - if (!btf_id) - return -EFAULT; - - if (prog->aux->func_info_aux[subprog].unreliable) - return -EINVAL; - - is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, false); - - /* Compiler optimizations can remove arguments from static functions - * or mismatched type can be passed into a global function. - * In such cases mark the function as unreliable from BTF point of view. - */ - if (err) - prog->aux->func_info_aux[subprog].unreliable = true; - return err; -} - -/* Compare BTF of a function call with given bpf_reg_state. - * Returns: - * EFAULT - there is a verifier bug. Abort verification. - * EINVAL - there is a type mismatch or BTF is not available. - * 0 - BTF matches with what bpf_reg_state expects. - * Only PTR_TO_CTX and SCALAR_VALUE states are recognized. - * - * NOTE: the code is duplicated from btf_check_subprog_arg_match() - * because btf_check_func_arg_match() is still doing both. Once that - * function is split in 2, we can call from here btf_check_subprog_arg_match() - * first, and then treat the calling part in a new code path. - */ -int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *regs) -{ - struct bpf_prog *prog = env->prog; - struct btf *btf = prog->aux->btf; - bool is_global; - u32 btf_id; - int err; - - if (!prog->aux->func_info) - return -EINVAL; - - btf_id = prog->aux->func_info[subprog].type_id; - if (!btf_id) - return -EFAULT; - - if (prog->aux->func_info_aux[subprog].unreliable) - return -EINVAL; - - is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, true); - - /* Compiler optimizations can remove arguments from static functions - * or mismatched type can be passed into a global function. - * In such cases mark the function as unreliable from BTF point of view. - */ - if (err) - prog->aux->func_info_aux[subprog].unreliable = true; - return err; + return false; } -/* Convert BTF of a function into bpf_reg_state if possible +/* Process BTF of a function to produce high-level expectation of function + * arguments (like ARG_PTR_TO_CTX, or ARG_PTR_TO_MEM, etc). This information + * is cached in subprog info for reuse. * Returns: * EFAULT - there is a verifier bug. Abort verification. * EINVAL - cannot convert BTF. - * 0 - Successfully converted BTF into bpf_reg_state - * (either PTR_TO_CTX or SCALAR_VALUE). + * 0 - Successfully processed BTF and constructed argument expectations. */ -int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, - struct bpf_reg_state *regs, u32 *arg_cnt) +int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) { + bool is_global = subprog_aux(env, subprog)->linkage == BTF_FUNC_GLOBAL; + struct bpf_subprog_info *sub = subprog_info(env, subprog); struct bpf_verifier_log *log = &env->log; struct bpf_prog *prog = env->prog; enum bpf_prog_type prog_type = prog->type; struct btf *btf = prog->aux->btf; const struct btf_param *args; - const struct btf_type *t, *ref_t; + const struct btf_type *t, *ref_t, *fn_t; u32 i, nargs, btf_id; const char *tname; - if (!prog->aux->func_info || - prog->aux->func_info_aux[subprog].linkage != BTF_FUNC_GLOBAL) { + if (sub->args_cached) + return 0; + + if (!prog->aux->func_info) { bpf_log(log, "Verifier bug\n"); return -EFAULT; } btf_id = prog->aux->func_info[subprog].type_id; if (!btf_id) { + if (!is_global) /* not fatal for static funcs */ + return -EINVAL; bpf_log(log, "Global functions need valid BTF\n"); return -EFAULT; } - t = btf_type_by_id(btf, btf_id); - if (!t || !btf_type_is_func(t)) { + fn_t = btf_type_by_id(btf, btf_id); + if (!fn_t || !btf_type_is_func(fn_t)) { /* These checks were already done by the verifier while loading * struct bpf_func_info */ @@ -6988,11 +6830,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, subprog); return -EFAULT; } - tname = btf_name_by_offset(btf, t->name_off); - - if (log->level & BPF_LOG_LEVEL) - bpf_log(log, "Validating %s() func#%d...\n", - tname, subprog); + tname = btf_name_by_offset(btf, fn_t->name_off); if (prog->aux->func_info_aux[subprog].unreliable) { bpf_log(log, "Verifier bug in function %s()\n", tname); @@ -7001,7 +6839,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, if (prog_type == BPF_PROG_TYPE_EXT) prog_type = prog->aux->dst_prog->type; - t = btf_type_by_id(btf, t->type); + t = btf_type_by_id(btf, fn_t->type); if (!t || !btf_type_is_func_proto(t)) { bpf_log(log, "Invalid type of function %s()\n", tname); return -EFAULT; @@ -7013,7 +6851,6 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, tname, nargs, MAX_BPF_FUNC_REG_ARGS); return -EINVAL; } - *arg_cnt = nargs; /* check that function returns int, exception cb also requires this */ t = btf_type_by_id(btf, t->type); while (btf_type_is_modifier(t)) @@ -7028,24 +6865,54 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, * Only PTR_TO_CTX and SCALAR are supported atm. */ for (i = 0; i < nargs; i++) { - struct bpf_reg_state *reg = ®s[i + 1]; + bool is_nonnull = false; + const char *tag; t = btf_type_by_id(btf, args[i].type); + + tag = btf_find_decl_tag_value(btf, fn_t, i, "arg:"); + if (IS_ERR(tag) && PTR_ERR(tag) == -ENOENT) { + tag = NULL; + } else if (IS_ERR(tag)) { + bpf_log(log, "arg#%d type's tag fetching failure: %ld\n", i, PTR_ERR(tag)); + return PTR_ERR(tag); + } + /* 'arg:<tag>' decl_tag takes precedence over derivation of + * register type from BTF type itself + */ + if (tag) { + /* disallow arg tags in static subprogs */ + if (!is_global) { + bpf_log(log, "arg#%d type tag is not supported in static functions\n", i); + return -EOPNOTSUPP; + } + if (strcmp(tag, "ctx") == 0) { + sub->args[i].arg_type = ARG_PTR_TO_CTX; + continue; + } + if (strcmp(tag, "nonnull") == 0) + is_nonnull = true; + } + while (btf_type_is_modifier(t)) t = btf_type_by_id(btf, t->type); if (btf_type_is_int(t) || btf_is_any_enum(t)) { - reg->type = SCALAR_VALUE; + sub->args[i].arg_type = ARG_ANYTHING; continue; } - if (btf_type_is_ptr(t)) { - if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { - reg->type = PTR_TO_CTX; - continue; - } + if (btf_type_is_ptr(t) && btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { + sub->args[i].arg_type = ARG_PTR_TO_CTX; + continue; + } + if (btf_type_is_ptr(t) && btf_is_dynptr_ptr(btf, t)) { + sub->args[i].arg_type = ARG_PTR_TO_DYNPTR | MEM_RDONLY; + continue; + } + if (is_global && btf_type_is_ptr(t)) { + u32 mem_size; t = btf_type_skip_modifiers(btf, t->type, NULL); - - ref_t = btf_resolve_size(btf, t, ®->mem_size); + ref_t = btf_resolve_size(btf, t, &mem_size); if (IS_ERR(ref_t)) { bpf_log(log, "arg#%d reference type('%s %s') size cannot be determined: %ld\n", @@ -7054,15 +6921,22 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, return -EINVAL; } - reg->type = PTR_TO_MEM | PTR_MAYBE_NULL; - reg->id = ++env->id_gen; - + sub->args[i].arg_type = is_nonnull ? ARG_PTR_TO_MEM : ARG_PTR_TO_MEM_OR_NULL; + sub->args[i].mem_size = mem_size; continue; } + if (is_nonnull) { + bpf_log(log, "arg#%d marked as non-null, but is not a pointer type\n", i); + return -EINVAL; + } bpf_log(log, "Arg#%d type %s in %s() is not supported yet.\n", i, btf_type_str(t), tname); return -EINVAL; } + + sub->arg_cnt = nargs; + sub->args_cached = true; + return 0; } diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c index 1aafb2ff2e95..41e0a55c35f5 100644 --- a/kernel/bpf/inode.c +++ b/kernel/bpf/inode.c @@ -599,8 +599,15 @@ EXPORT_SYMBOL(bpf_prog_get_type_path); */ static int bpf_show_options(struct seq_file *m, struct dentry *root) { - umode_t mode = d_inode(root)->i_mode & S_IALLUGO & ~S_ISVTX; - + struct inode *inode = d_inode(root); + umode_t mode = inode->i_mode & S_IALLUGO & ~S_ISVTX; + + if (!uid_eq(inode->i_uid, GLOBAL_ROOT_UID)) + seq_printf(m, ",uid=%u", + from_kuid_munged(&init_user_ns, inode->i_uid)); + if (!gid_eq(inode->i_gid, GLOBAL_ROOT_GID)) + seq_printf(m, ",gid=%u", + from_kgid_munged(&init_user_ns, inode->i_gid)); if (mode != S_IRWXUGO) seq_printf(m, ",mode=%o", mode); return 0; @@ -625,15 +632,21 @@ static const struct super_operations bpf_super_ops = { }; enum { + OPT_UID, + OPT_GID, OPT_MODE, }; static const struct fs_parameter_spec bpf_fs_parameters[] = { + fsparam_u32 ("uid", OPT_UID), + fsparam_u32 ("gid", OPT_GID), fsparam_u32oct ("mode", OPT_MODE), {} }; struct bpf_mount_opts { + kuid_t uid; + kgid_t gid; umode_t mode; }; @@ -641,6 +654,8 @@ static int bpf_parse_param(struct fs_context *fc, struct fs_parameter *param) { struct bpf_mount_opts *opts = fc->fs_private; struct fs_parse_result result; + kuid_t uid; + kgid_t gid; int opt; opt = fs_parse(fc, bpf_fs_parameters, param, &result); @@ -662,12 +677,42 @@ static int bpf_parse_param(struct fs_context *fc, struct fs_parameter *param) } switch (opt) { + case OPT_UID: + uid = make_kuid(current_user_ns(), result.uint_32); + if (!uid_valid(uid)) + goto bad_value; + + /* + * The requested uid must be representable in the + * filesystem's idmapping. + */ + if (!kuid_has_mapping(fc->user_ns, uid)) + goto bad_value; + + opts->uid = uid; + break; + case OPT_GID: + gid = make_kgid(current_user_ns(), result.uint_32); + if (!gid_valid(gid)) + goto bad_value; + + /* + * The requested gid must be representable in the + * filesystem's idmapping. + */ + if (!kgid_has_mapping(fc->user_ns, gid)) + goto bad_value; + + opts->gid = gid; + break; case OPT_MODE: opts->mode = result.uint_32 & S_IALLUGO; break; } return 0; +bad_value: + return invalfc(fc, "Bad value for '%s'", param->key); } struct bpf_preload_ops *bpf_preload_ops; @@ -750,6 +795,8 @@ static int bpf_fill_super(struct super_block *sb, struct fs_context *fc) sb->s_op = &bpf_super_ops; inode = sb->s_root->d_inode; + inode->i_uid = opts->uid; + inode->i_gid = opts->gid; inode->i_op = &bpf_dir_iops; inode->i_mode &= ~S_IALLUGO; populate_bpffs(sb->s_root); @@ -785,6 +832,8 @@ static int bpf_init_fs_context(struct fs_context *fc) return -ENOMEM; opts->mode = S_IRWXUGO; + opts->uid = current_fsuid(); + opts->gid = current_fsgid(); fc->fs_private = opts; fc->ops = &bpf_context_ops; diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c index 6a51cfe4c2d6..aa0fbf000a12 100644 --- a/kernel/bpf/memalloc.c +++ b/kernel/bpf/memalloc.c @@ -490,27 +490,6 @@ static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) alloc_bulk(c, c->unit_size <= 256 ? 4 : 1, cpu_to_node(cpu), false); } -static int check_obj_size(struct bpf_mem_cache *c, unsigned int idx) -{ - struct llist_node *first; - unsigned int obj_size; - - first = c->free_llist.first; - if (!first) - return 0; - - if (c->percpu_size) - obj_size = pcpu_alloc_size(((void **)first)[1]); - else - obj_size = ksize(first); - if (obj_size != c->unit_size) { - WARN_ONCE(1, "bpf_mem_cache[%u]: percpu %d, unexpected object size %u, expect %u\n", - idx, c->percpu_size, obj_size, c->unit_size); - return -EINVAL; - } - return 0; -} - /* When size != 0 bpf_mem_cache for each cpu. * This is typical bpf hash map use case when all elements have equal size. * @@ -521,10 +500,10 @@ static int check_obj_size(struct bpf_mem_cache *c, unsigned int idx) int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) { static u16 sizes[NUM_CACHES] = {96, 192, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096}; - int cpu, i, err, unit_size, percpu_size = 0; struct bpf_mem_caches *cc, __percpu *pcc; struct bpf_mem_cache *c, __percpu *pc; struct obj_cgroup *objcg = NULL; + int cpu, i, unit_size, percpu_size = 0; /* room for llist_node and per-cpu pointer */ if (percpu) @@ -560,7 +539,6 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) pcc = __alloc_percpu_gfp(sizeof(*cc), 8, GFP_KERNEL); if (!pcc) return -ENOMEM; - err = 0; #ifdef CONFIG_MEMCG_KMEM objcg = get_obj_cgroup_from_current(); #endif @@ -574,28 +552,12 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) c->tgt = c; init_refill_work(c); - /* Another bpf_mem_cache will be used when allocating - * c->unit_size in bpf_mem_alloc(), so doesn't prefill - * for the bpf_mem_cache because these free objects will - * never be used. - */ - if (i != bpf_mem_cache_idx(c->unit_size)) - continue; prefill_mem_cache(c, cpu); - err = check_obj_size(c, i); - if (err) - goto out; } } -out: ma->caches = pcc; - /* refill_work is either zeroed or initialized, so it is safe to - * call irq_work_sync(). - */ - if (err) - bpf_mem_alloc_destroy(ma); - return err; + return 0; } static void drain_mem_cache(struct bpf_mem_cache *c) @@ -869,7 +831,7 @@ void notrace *bpf_mem_alloc(struct bpf_mem_alloc *ma, size_t size) void *ret; if (!size) - return ZERO_SIZE_PTR; + return NULL; idx = bpf_mem_cache_idx(size + LLIST_NODE_SZ); if (idx < 0) @@ -879,26 +841,17 @@ void notrace *bpf_mem_alloc(struct bpf_mem_alloc *ma, size_t size) return !ret ? NULL : ret + LLIST_NODE_SZ; } -static notrace int bpf_mem_free_idx(void *ptr, bool percpu) -{ - size_t size; - - if (percpu) - size = pcpu_alloc_size(*((void **)ptr)); - else - size = ksize(ptr - LLIST_NODE_SZ); - return bpf_mem_cache_idx(size); -} - void notrace bpf_mem_free(struct bpf_mem_alloc *ma, void *ptr) { + struct bpf_mem_cache *c; int idx; if (!ptr) return; - idx = bpf_mem_free_idx(ptr, ma->percpu); - if (idx < 0) + c = *(void **)(ptr - LLIST_NODE_SZ); + idx = bpf_mem_cache_idx(c->unit_size); + if (WARN_ON_ONCE(idx < 0)) return; unit_free(this_cpu_ptr(ma->caches)->cache + idx, ptr); @@ -906,13 +859,15 @@ void notrace bpf_mem_free(struct bpf_mem_alloc *ma, void *ptr) void notrace bpf_mem_free_rcu(struct bpf_mem_alloc *ma, void *ptr) { + struct bpf_mem_cache *c; int idx; if (!ptr) return; - idx = bpf_mem_free_idx(ptr, ma->percpu); - if (idx < 0) + c = *(void **)(ptr - LLIST_NODE_SZ); + idx = bpf_mem_cache_idx(c->unit_size); + if (WARN_ON_ONCE(idx < 0)) return; unit_free_rcu(this_cpu_ptr(ma->caches)->cache + idx, ptr); @@ -986,41 +941,3 @@ void notrace *bpf_mem_cache_alloc_flags(struct bpf_mem_alloc *ma, gfp_t flags) return !ret ? NULL : ret + LLIST_NODE_SZ; } - -/* The alignment of dynamic per-cpu area is 8, so c->unit_size and the - * actual size of dynamic per-cpu area will always be matched and there is - * no need to adjust size_index for per-cpu allocation. However for the - * simplicity of the implementation, use an unified size_index for both - * kmalloc and per-cpu allocation. - */ -static __init int bpf_mem_cache_adjust_size(void) -{ - unsigned int size; - - /* Adjusting the indexes in size_index() according to the object_size - * of underlying slab cache, so bpf_mem_alloc() will select a - * bpf_mem_cache with unit_size equal to the object_size of - * the underlying slab cache. - * - * The maximal value of KMALLOC_MIN_SIZE and __kmalloc_minalign() is - * 256-bytes, so only do adjustment for [8-bytes, 192-bytes]. - */ - for (size = 192; size >= 8; size -= 8) { - unsigned int kmalloc_size, index; - - kmalloc_size = kmalloc_size_roundup(size); - if (kmalloc_size == size) - continue; - - if (kmalloc_size <= 192) - index = size_index[(kmalloc_size - 1) / 8]; - else - index = fls(kmalloc_size - 1) - 1; - /* Only overwrite if necessary */ - if (size_index[(size - 1) / 8] != index) - size_index[(size - 1) / 8] = index; - } - - return 0; -} -subsys_initcall(bpf_mem_cache_adjust_size); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 4ceec8c2a484..a376eb609c41 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -437,16 +437,6 @@ static const char *subprog_name(const struct bpf_verifier_env *env, int subprog) return btf_type_name(env->prog->aux->btf, info->type_id); } -static struct bpf_func_info_aux *subprog_aux(const struct bpf_verifier_env *env, int subprog) -{ - return &env->prog->aux->func_info_aux[subprog]; -} - -static struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env, int subprog) -{ - return &env->subprog_info[subprog]; -} - static void mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog) { struct bpf_subprog_info *info = subprog_info(env, subprog); @@ -5137,8 +5127,8 @@ static int __check_ptr_off_reg(struct bpf_verifier_env *env, return 0; } -int check_ptr_off_reg(struct bpf_verifier_env *env, - const struct bpf_reg_state *reg, int regno) +static int check_ptr_off_reg(struct bpf_verifier_env *env, + const struct bpf_reg_state *reg, int regno) { return __check_ptr_off_reg(env, reg, regno, false); } @@ -7310,8 +7300,8 @@ static int check_mem_size_reg(struct bpf_verifier_env *env, return err; } -int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno, u32 mem_size) +static int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, + u32 regno, u32 mem_size) { bool may_be_null = type_may_be_null(reg->type); struct bpf_reg_state saved_reg; @@ -8296,9 +8286,9 @@ reg_find_field_offset(const struct bpf_reg_state *reg, s32 off, u32 fields) return field; } -int check_func_arg_reg_off(struct bpf_verifier_env *env, - const struct bpf_reg_state *reg, int regno, - enum bpf_arg_type arg_type) +static int check_func_arg_reg_off(struct bpf_verifier_env *env, + const struct bpf_reg_state *reg, int regno, + enum bpf_arg_type arg_type) { u32 type = reg->type; @@ -9259,6 +9249,102 @@ err_out: return err; } +static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog, + const struct btf *btf, + struct bpf_reg_state *regs) +{ + struct bpf_subprog_info *sub = subprog_info(env, subprog); + struct bpf_verifier_log *log = &env->log; + u32 i; + int ret; + + ret = btf_prepare_func_args(env, subprog); + if (ret) + return ret; + + /* check that BTF function arguments match actual types that the + * verifier sees. + */ + for (i = 0; i < sub->arg_cnt; i++) { + u32 regno = i + 1; + struct bpf_reg_state *reg = ®s[regno]; + struct bpf_subprog_arg_info *arg = &sub->args[i]; + + if (arg->arg_type == ARG_ANYTHING) { + if (reg->type != SCALAR_VALUE) { + bpf_log(log, "R%d is not a scalar\n", regno); + return -EINVAL; + } + } else if (arg->arg_type == ARG_PTR_TO_CTX) { + ret = check_func_arg_reg_off(env, reg, regno, ARG_DONTCARE); + if (ret < 0) + return ret; + /* If function expects ctx type in BTF check that caller + * is passing PTR_TO_CTX. + */ + if (reg->type != PTR_TO_CTX) { + bpf_log(log, "arg#%d expects pointer to ctx\n", i); + return -EINVAL; + } + } else if (base_type(arg->arg_type) == ARG_PTR_TO_MEM) { + ret = check_func_arg_reg_off(env, reg, regno, ARG_DONTCARE); + if (ret < 0) + return ret; + if (check_mem_reg(env, reg, regno, arg->mem_size)) + return -EINVAL; + if (!(arg->arg_type & PTR_MAYBE_NULL) && (reg->type & PTR_MAYBE_NULL)) { + bpf_log(log, "arg#%d is expected to be non-NULL\n", i); + return -EINVAL; + } + } else if (arg->arg_type == (ARG_PTR_TO_DYNPTR | MEM_RDONLY)) { + ret = process_dynptr_func(env, regno, -1, arg->arg_type, 0); + if (ret) + return ret; + } else { + bpf_log(log, "verifier bug: unrecognized arg#%d type %d\n", + i, arg->arg_type); + return -EFAULT; + } + } + + return 0; +} + +/* Compare BTF of a function call with given bpf_reg_state. + * Returns: + * EFAULT - there is a verifier bug. Abort verification. + * EINVAL - there is a type mismatch or BTF is not available. + * 0 - BTF matches with what bpf_reg_state expects. + * Only PTR_TO_CTX and SCALAR_VALUE states are recognized. + */ +static int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, + struct bpf_reg_state *regs) +{ + struct bpf_prog *prog = env->prog; + struct btf *btf = prog->aux->btf; + u32 btf_id; + int err; + + if (!prog->aux->func_info) + return -EINVAL; + + btf_id = prog->aux->func_info[subprog].type_id; + if (!btf_id) + return -EFAULT; + + if (prog->aux->func_info_aux[subprog].unreliable) + return -EINVAL; + + err = btf_check_func_arg_match(env, subprog, btf, regs); + /* Compiler optimizations can remove arguments from static functions + * or mismatched type can be passed into a global function. + * In such cases mark the function as unreliable from BTF point of view. + */ + if (err) + prog->aux->func_info_aux[subprog].unreliable = true; + return err; +} + static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int insn_idx, int subprog, set_callee_state_fn set_callee_state_cb) @@ -9530,7 +9616,7 @@ static int set_find_vma_callback_state(struct bpf_verifier_env *env, callee->regs[BPF_REG_2].type = PTR_TO_BTF_ID; __mark_reg_known_zero(&callee->regs[BPF_REG_2]); callee->regs[BPF_REG_2].btf = btf_vmlinux; - callee->regs[BPF_REG_2].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_VMA], + callee->regs[BPF_REG_2].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_VMA]; /* pointer to stack or null */ callee->regs[BPF_REG_3] = caller->regs[BPF_REG_4]; @@ -14336,7 +14422,43 @@ again: } break; case BPF_JNE: - /* we don't derive any new information for inequality yet */ + if (!is_reg_const(reg2, is_jmp32)) + swap(reg1, reg2); + if (!is_reg_const(reg2, is_jmp32)) + break; + + /* try to recompute the bound of reg1 if reg2 is a const and + * is exactly the edge of reg1. + */ + val = reg_const_value(reg2, is_jmp32); + if (is_jmp32) { + /* u32_min_value is not equal to 0xffffffff at this point, + * because otherwise u32_max_value is 0xffffffff as well, + * in such a case both reg1 and reg2 would be constants, + * jump would be predicted and reg_set_min_max() won't + * be called. + * + * Same reasoning works for all {u,s}{min,max}{32,64} cases + * below. + */ + if (reg1->u32_min_value == (u32)val) + reg1->u32_min_value++; + if (reg1->u32_max_value == (u32)val) + reg1->u32_max_value--; + if (reg1->s32_min_value == (s32)val) + reg1->s32_min_value++; + if (reg1->s32_max_value == (s32)val) + reg1->s32_max_value--; + } else { + if (reg1->umin_value == (u64)val) + reg1->umin_value++; + if (reg1->umax_value == (u64)val) + reg1->umax_value--; + if (reg1->smin_value == (s64)val) + reg1->smin_value++; + if (reg1->smax_value == (s64)val) + reg1->smax_value--; + } break; case BPF_JSET: if (!is_reg_const(reg2, is_jmp32)) @@ -19873,6 +19995,7 @@ static void free_states(struct bpf_verifier_env *env) static int do_check_common(struct bpf_verifier_env *env, int subprog) { bool pop_log = !(env->log.level & BPF_LOG_LEVEL2); + struct bpf_subprog_info *sub = subprog_info(env, subprog); struct bpf_verifier_state *state; struct bpf_reg_state *regs; int ret, i; @@ -19899,54 +20022,71 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) state->first_insn_idx = env->subprog_info[subprog].start; state->last_insn_idx = -1; + regs = state->frame[state->curframe]->regs; if (subprog || env->prog->type == BPF_PROG_TYPE_EXT) { - u32 nargs; + const char *sub_name = subprog_name(env, subprog); + struct bpf_subprog_arg_info *arg; + struct bpf_reg_state *reg; - ret = btf_prepare_func_args(env, subprog, regs, &nargs); + verbose(env, "Validating %s() func#%d...\n", sub_name, subprog); + ret = btf_prepare_func_args(env, subprog); if (ret) goto out; + if (subprog_is_exc_cb(env, subprog)) { state->frame[0]->in_exception_callback_fn = true; /* We have already ensured that the callback returns an integer, just * like all global subprogs. We need to determine it only has a single * scalar argument. */ - if (nargs != 1 || regs[BPF_REG_1].type != SCALAR_VALUE) { + if (sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_ANYTHING) { verbose(env, "exception cb only supports single integer argument\n"); ret = -EINVAL; goto out; } } - for (i = BPF_REG_1; i <= BPF_REG_5; i++) { - if (regs[i].type == PTR_TO_CTX) + for (i = BPF_REG_1; i <= sub->arg_cnt; i++) { + arg = &sub->args[i - BPF_REG_1]; + reg = ®s[i]; + + if (arg->arg_type == ARG_PTR_TO_CTX) { + reg->type = PTR_TO_CTX; mark_reg_known_zero(env, regs, i); - else if (regs[i].type == SCALAR_VALUE) + } else if (arg->arg_type == ARG_ANYTHING) { + reg->type = SCALAR_VALUE; mark_reg_unknown(env, regs, i); - else if (base_type(regs[i].type) == PTR_TO_MEM) { - const u32 mem_size = regs[i].mem_size; - + } else if (arg->arg_type == (ARG_PTR_TO_DYNPTR | MEM_RDONLY)) { + /* assume unspecial LOCAL dynptr type */ + __mark_dynptr_reg(reg, BPF_DYNPTR_TYPE_LOCAL, true, ++env->id_gen); + } else if (base_type(arg->arg_type) == ARG_PTR_TO_MEM) { + reg->type = PTR_TO_MEM; + if (arg->arg_type & PTR_MAYBE_NULL) + reg->type |= PTR_MAYBE_NULL; mark_reg_known_zero(env, regs, i); - regs[i].mem_size = mem_size; - regs[i].id = ++env->id_gen; + reg->mem_size = arg->mem_size; + reg->id = ++env->id_gen; + } else { + WARN_ONCE(1, "BUG: unhandled arg#%d type %d\n", + i - BPF_REG_1, arg->arg_type); + ret = -EFAULT; + goto out; } } } else { + /* if main BPF program has associated BTF info, validate that + * it's matching expected signature, and otherwise mark BTF + * info for main program as unreliable + */ + if (env->prog->aux->func_info_aux) { + ret = btf_prepare_func_args(env, 0); + if (ret || sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_PTR_TO_CTX) + env->prog->aux->func_info_aux[0].unreliable = true; + } + /* 1st arg to a function */ regs[BPF_REG_1].type = PTR_TO_CTX; mark_reg_known_zero(env, regs, BPF_REG_1); - ret = btf_check_subprog_arg_match(env, subprog, regs); - if (ret == -EFAULT) - /* unlikely verifier bug. abort. - * ret == 0 and ret < 0 are sadly acceptable for - * main() function due to backward compatibility. - * Like socket filter program may be written as: - * int bpf_prog(struct pt_regs *ctx) - * and never dereference that ctx in the program. - * 'struct pt_regs' is a type mismatch for socket - * filter that should be using 'struct __sk_buff'. - */ - goto out; } ret = do_check(env); diff --git a/net/appletalk/ddp.c b/net/appletalk/ddp.c index a852ec093fa8..198f5ba2feae 100644 --- a/net/appletalk/ddp.c +++ b/net/appletalk/ddp.c @@ -1581,7 +1581,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) } /* Build a packet */ - SOCK_DEBUG(sk, "SK %p: Got address.\n", sk); + net_dbg_ratelimited("SK %p: Got address.\n", sk); /* For headers */ size = sizeof(struct ddpehdr) + len + ddp_dl->header_length; @@ -1602,7 +1602,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) dev = rt->dev; - SOCK_DEBUG(sk, "SK %p: Size needed %d, device %s\n", + net_dbg_ratelimited("SK %p: Size needed %d, device %s\n", sk, size, dev->name); hard_header_len = dev->hard_header_len; @@ -1631,7 +1631,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) skb_reserve(skb, hard_header_len); skb->dev = dev; - SOCK_DEBUG(sk, "SK %p: Begin build.\n", sk); + net_dbg_ratelimited("SK %p: Begin build.\n", sk); ddp = skb_put(skb, sizeof(struct ddpehdr)); ddp->deh_len_hops = htons(len + sizeof(*ddp)); @@ -1642,7 +1642,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) ddp->deh_dport = usat->sat_port; ddp->deh_sport = at->src_port; - SOCK_DEBUG(sk, "SK %p: Copy user data (%zd bytes).\n", sk, len); + net_dbg_ratelimited("SK %p: Copy user data (%zd bytes).\n", sk, len); err = memcpy_from_msg(skb_put(skb, len), msg, len); if (err) { @@ -1666,7 +1666,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) if (skb2) { loopback = 1; - SOCK_DEBUG(sk, "SK %p: send out(copy).\n", sk); + net_dbg_ratelimited("SK %p: send out(copy).\n", sk); /* * If it fails it is queued/sent above in the aarp queue */ @@ -1675,7 +1675,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) } if (dev->flags & IFF_LOOPBACK || loopback) { - SOCK_DEBUG(sk, "SK %p: Loop back.\n", sk); + net_dbg_ratelimited("SK %p: Loop back.\n", sk); /* loop back */ skb_orphan(skb); if (ddp->deh_dnode == ATADDR_BCAST) { @@ -1689,7 +1689,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) } ddp_dl->request(ddp_dl, skb, dev->dev_addr); } else { - SOCK_DEBUG(sk, "SK %p: send out.\n", sk); + net_dbg_ratelimited("SK %p: send out.\n", sk); if (rt->flags & RTF_GATEWAY) { gsat.sat_addr = rt->gateway; usat = &gsat; @@ -1700,7 +1700,7 @@ static int atalk_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) */ aarp_send_ddp(dev, skb, &usat->sat_addr, NULL); } - SOCK_DEBUG(sk, "SK %p: Done write (%zd).\n", sk, len); + net_dbg_ratelimited("SK %p: Done write (%zd).\n", sk, len); out: release_sock(sk); diff --git a/net/bridge/br_cfm_netlink.c b/net/bridge/br_cfm_netlink.c index 5c4c369f8536..2faab44652e7 100644 --- a/net/bridge/br_cfm_netlink.c +++ b/net/bridge/br_cfm_netlink.c @@ -362,7 +362,7 @@ static int br_cc_ccm_tx_parse(struct net_bridge *br, struct nlattr *attr, memset(&tx_info, 0, sizeof(tx_info)); - instance = nla_get_u32(tb[IFLA_BRIDGE_CFM_CC_RDI_INSTANCE]); + instance = nla_get_u32(tb[IFLA_BRIDGE_CFM_CC_CCM_TX_INSTANCE]); nla_memcpy(&tx_info.dmac.addr, tb[IFLA_BRIDGE_CFM_CC_CCM_TX_DMAC], sizeof(tx_info.dmac.addr)); diff --git a/net/core/dev.c b/net/core/dev.c index f9d4b550ef4b..31588a50b757 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -153,6 +153,7 @@ #include <linux/prandom.h> #include <linux/once_lite.h> #include <net/netdev_rx_queue.h> +#include <linux/phy_link_topology_core.h> #include "dev.h" #include "net-sysfs.h" @@ -10875,6 +10876,8 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, #ifdef CONFIG_NET_SCHED hash_init(dev->qdisc_hash); #endif + phy_link_topo_init(&dev->link_topo); + dev->priv_flags = IFF_XMIT_DST_RELEASE | IFF_XMIT_DST_RELEASE_PERM; setup(dev); @@ -11626,6 +11629,7 @@ static void __init net_dev_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_ipv4_max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_max_segs); + CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, gso_partial_features); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, num_tc); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, mtu); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, needed_headroom); @@ -11639,7 +11643,7 @@ static void __init net_dev_struct_check(void) #ifdef CONFIG_NET_XGRESS CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_tx, tcx_egress); #endif - CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_tx, 152); + CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_tx, 160); /* TXRX read-mostly hotpath */ CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_txrx, flags); diff --git a/net/core/drop_monitor.c b/net/core/drop_monitor.c index b240d9aae4a6..b0f221d658be 100644 --- a/net/core/drop_monitor.c +++ b/net/core/drop_monitor.c @@ -183,7 +183,7 @@ out: } static const struct genl_multicast_group dropmon_mcgrps[] = { - { .name = "events", .cap_sys_admin = 1 }, + { .name = "events", .flags = GENL_MCAST_CAP_SYS_ADMIN, }, }; static void send_dm_alert(struct work_struct *work) diff --git a/net/core/pktgen.c b/net/core/pktgen.c index 57cea67b7562..ea55a758a475 100644 --- a/net/core/pktgen.c +++ b/net/core/pktgen.c @@ -3669,10 +3669,8 @@ static int pktgen_thread_worker(void *arg) if (unlikely(!pkt_dev && t->control == 0)) { if (t->net->pktgen_exiting) break; - wait_event_interruptible_timeout(t->queue, - t->control != 0, - HZ/10); - try_to_freeze(); + wait_event_freezable_timeout(t->queue, + t->control != 0, HZ / 10); continue; } diff --git a/net/core/skbuff.c b/net/core/skbuff.c index ce5687ddb768..12d22c0b8551 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5995,6 +5995,31 @@ int skb_ensure_writable(struct sk_buff *skb, unsigned int write_len) } EXPORT_SYMBOL(skb_ensure_writable); +int skb_ensure_writable_head_tail(struct sk_buff *skb, struct net_device *dev) +{ + int needed_headroom = dev->needed_headroom; + int needed_tailroom = dev->needed_tailroom; + + /* For tail taggers, we need to pad short frames ourselves, to ensure + * that the tail tag does not fail at its role of being at the end of + * the packet, once the conduit interface pads the frame. Account for + * that pad length here, and pad later. + */ + if (unlikely(needed_tailroom && skb->len < ETH_ZLEN)) + needed_tailroom += ETH_ZLEN - skb->len; + /* skb_headroom() returns unsigned int... */ + needed_headroom = max_t(int, needed_headroom - skb_headroom(skb), 0); + needed_tailroom = max_t(int, needed_tailroom - skb_tailroom(skb), 0); + + if (likely(!needed_headroom && !needed_tailroom && !skb_cloned(skb))) + /* No reallocation needed, yay! */ + return 0; + + return pskb_expand_head(skb, needed_headroom, needed_tailroom, + GFP_ATOMIC); +} +EXPORT_SYMBOL(skb_ensure_writable_head_tail); + /* remove VLAN header from packet and update csum accordingly. * expects a non skb_vlan_tag_present skb with a vlan tag payload */ diff --git a/net/dccp/ipv6.c b/net/dccp/ipv6.c index 06d7324276ec..ded07e09f813 100644 --- a/net/dccp/ipv6.c +++ b/net/dccp/ipv6.c @@ -889,7 +889,7 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, u32 exthdrlen = icsk->icsk_ext_hdr_len; struct sockaddr_in sin; - SOCK_DEBUG(sk, "connect: ipv4 mapped\n"); + net_dbg_ratelimited("connect: ipv4 mapped\n"); if (ipv6_only_sock(sk)) return -ENETUNREACH; diff --git a/net/dsa/user.c b/net/dsa/user.c index d438884a4eb0..b738a466e2dc 100644 --- a/net/dsa/user.c +++ b/net/dsa/user.c @@ -920,30 +920,6 @@ netdev_tx_t dsa_enqueue_skb(struct sk_buff *skb, struct net_device *dev) } EXPORT_SYMBOL_GPL(dsa_enqueue_skb); -static int dsa_realloc_skb(struct sk_buff *skb, struct net_device *dev) -{ - int needed_headroom = dev->needed_headroom; - int needed_tailroom = dev->needed_tailroom; - - /* For tail taggers, we need to pad short frames ourselves, to ensure - * that the tail tag does not fail at its role of being at the end of - * the packet, once the conduit interface pads the frame. Account for - * that pad length here, and pad later. - */ - if (unlikely(needed_tailroom && skb->len < ETH_ZLEN)) - needed_tailroom += ETH_ZLEN - skb->len; - /* skb_headroom() returns unsigned int... */ - needed_headroom = max_t(int, needed_headroom - skb_headroom(skb), 0); - needed_tailroom = max_t(int, needed_tailroom - skb_tailroom(skb), 0); - - if (likely(!needed_headroom && !needed_tailroom && !skb_cloned(skb))) - /* No reallocation needed, yay! */ - return 0; - - return pskb_expand_head(skb, needed_headroom, needed_tailroom, - GFP_ATOMIC); -} - static netdev_tx_t dsa_user_xmit(struct sk_buff *skb, struct net_device *dev) { struct dsa_user_priv *p = netdev_priv(dev); @@ -956,13 +932,14 @@ static netdev_tx_t dsa_user_xmit(struct sk_buff *skb, struct net_device *dev) /* Handle tx timestamp if any */ dsa_skb_tx_timestamp(p, skb); - if (dsa_realloc_skb(skb, dev)) { + if (skb_ensure_writable_head_tail(skb, dev)) { dev_kfree_skb_any(skb); return NETDEV_TX_OK; } /* needed_tailroom should still be 'warm' in the cache line from - * dsa_realloc_skb(), which has also ensured that padding is safe. + * skb_ensure_writable_head_tail(), which has also ensured that + * padding is safe. */ if (dev->needed_tailroom) eth_skb_pad(skb); diff --git a/net/ethtool/Makefile b/net/ethtool/Makefile index 504f954a1b28..0ccd0e9afd3f 100644 --- a/net/ethtool/Makefile +++ b/net/ethtool/Makefile @@ -8,4 +8,4 @@ ethtool_nl-y := netlink.o bitset.o strset.o linkinfo.o linkmodes.o rss.o \ linkstate.o debug.o wol.o features.o privflags.o rings.o \ channels.o coalesce.o pause.o eee.o tsinfo.o cabletest.o \ tunnels.o fec.o eeprom.o stats.o phc_vclocks.o mm.o \ - module.o pse-pd.o plca.o mm.o + module.o pse-pd.o plca.o mm.o phy.o diff --git a/net/ethtool/cabletest.c b/net/ethtool/cabletest.c index 06a151165c31..6b00d0800f23 100644 --- a/net/ethtool/cabletest.c +++ b/net/ethtool/cabletest.c @@ -69,7 +69,7 @@ int ethnl_act_cable_test(struct sk_buff *skb, struct genl_info *info) return ret; dev = req_info.dev; - if (!dev->phydev) { + if (!req_info.phydev) { ret = -EOPNOTSUPP; goto out_dev_put; } @@ -85,12 +85,12 @@ int ethnl_act_cable_test(struct sk_buff *skb, struct genl_info *info) if (ret < 0) goto out_rtnl; - ret = ops->start_cable_test(dev->phydev, info->extack); + ret = ops->start_cable_test(req_info.phydev, info->extack); ethnl_ops_complete(dev); if (!ret) - ethnl_cable_test_started(dev->phydev, + ethnl_cable_test_started(req_info.phydev, ETHTOOL_MSG_CABLE_TEST_NTF); out_rtnl: @@ -321,7 +321,7 @@ int ethnl_act_cable_test_tdr(struct sk_buff *skb, struct genl_info *info) return ret; dev = req_info.dev; - if (!dev->phydev) { + if (!req_info.phydev) { ret = -EOPNOTSUPP; goto out_dev_put; } @@ -342,12 +342,12 @@ int ethnl_act_cable_test_tdr(struct sk_buff *skb, struct genl_info *info) if (ret < 0) goto out_rtnl; - ret = ops->start_cable_test_tdr(dev->phydev, info->extack, &cfg); + ret = ops->start_cable_test_tdr(req_info.phydev, info->extack, &cfg); ethnl_ops_complete(dev); if (!ret) - ethnl_cable_test_started(dev->phydev, + ethnl_cable_test_started(req_info.phydev, ETHTOOL_MSG_CABLE_TEST_TDR_NTF); out_rtnl: diff --git a/net/ethtool/netlink.c b/net/ethtool/netlink.c index fe3553f60bf3..92b0dd8ca046 100644 --- a/net/ethtool/netlink.c +++ b/net/ethtool/netlink.c @@ -4,6 +4,7 @@ #include <linux/ethtool_netlink.h> #include <linux/pm_runtime.h> #include "netlink.h" +#include <linux/phy_link_topology.h> static struct genl_family ethtool_genl_family; @@ -20,6 +21,7 @@ const struct nla_policy ethnl_header_policy[] = { .len = ALTIFNAMSIZ - 1 }, [ETHTOOL_A_HEADER_FLAGS] = NLA_POLICY_MASK(NLA_U32, ETHTOOL_FLAGS_BASIC), + [ETHTOOL_A_HEADER_PHY_INDEX] = NLA_POLICY_MIN(NLA_U32, 1), }; const struct nla_policy ethnl_header_policy_stats[] = { @@ -28,6 +30,7 @@ const struct nla_policy ethnl_header_policy_stats[] = { .len = ALTIFNAMSIZ - 1 }, [ETHTOOL_A_HEADER_FLAGS] = NLA_POLICY_MASK(NLA_U32, ETHTOOL_FLAGS_STATS), + [ETHTOOL_A_HEADER_PHY_INDEX] = NLA_POLICY_MIN(NLA_U32, 1), }; int ethnl_ops_begin(struct net_device *dev) @@ -91,6 +94,7 @@ int ethnl_parse_header_dev_get(struct ethnl_req_info *req_info, { struct nlattr *tb[ARRAY_SIZE(ethnl_header_policy)]; const struct nlattr *devname_attr; + struct phy_device *phydev = NULL; struct net_device *dev = NULL; u32 flags = 0; int ret; @@ -145,6 +149,26 @@ int ethnl_parse_header_dev_get(struct ethnl_req_info *req_info, return -EINVAL; } + if (dev) { + if (tb[ETHTOOL_A_HEADER_PHY_INDEX]) { + u32 phy_index = nla_get_u32(tb[ETHTOOL_A_HEADER_PHY_INDEX]); + + phydev = phy_link_topo_get_phy(&dev->link_topo, + phy_index); + if (!phydev) { + NL_SET_ERR_MSG_ATTR(extack, header, + "no phy matches phy index"); + return -EINVAL; + } + } else { + /* If we need a PHY but no phy index is specified, fallback + * to dev->phydev + */ + phydev = dev->phydev; + } + } + + req_info->phydev = phydev; req_info->dev = dev; req_info->flags = flags; return 0; @@ -1129,6 +1153,15 @@ static const struct genl_ops ethtool_genl_ops[] = { .policy = ethnl_mm_set_policy, .maxattr = ARRAY_SIZE(ethnl_mm_set_policy) - 1, }, + { + .cmd = ETHTOOL_MSG_PHY_GET, + .doit = ethnl_phy_doit, + .start = ethnl_phy_start, + .dumpit = ethnl_phy_dumpit, + .done = ethnl_phy_done, + .policy = ethnl_phy_get_policy, + .maxattr = ARRAY_SIZE(ethnl_phy_get_policy) - 1, + }, }; static const struct genl_multicast_group ethtool_nl_mcgrps[] = { diff --git a/net/ethtool/netlink.h b/net/ethtool/netlink.h index 9a333a8d04c1..5e6a43e35a09 100644 --- a/net/ethtool/netlink.h +++ b/net/ethtool/netlink.h @@ -250,6 +250,7 @@ static inline unsigned int ethnl_reply_header_size(void) * @dev: network device the request is for (may be null) * @dev_tracker: refcount tracker for @dev reference * @flags: request flags common for all request types + * @phydev: phy_device connected to @dev this request is for (may be null) * * This is a common base for request specific structures holding data from * parsed userspace request. These always embed struct ethnl_req_info at @@ -259,6 +260,7 @@ struct ethnl_req_info { struct net_device *dev; netdevice_tracker dev_tracker; u32 flags; + struct phy_device *phydev; }; static inline void ethnl_parse_header_dev_put(struct ethnl_req_info *req_info) @@ -395,9 +397,10 @@ extern const struct ethnl_request_ops ethnl_rss_request_ops; extern const struct ethnl_request_ops ethnl_plca_cfg_request_ops; extern const struct ethnl_request_ops ethnl_plca_status_request_ops; extern const struct ethnl_request_ops ethnl_mm_request_ops; +extern const struct ethnl_request_ops ethnl_phy_request_ops; -extern const struct nla_policy ethnl_header_policy[ETHTOOL_A_HEADER_FLAGS + 1]; -extern const struct nla_policy ethnl_header_policy_stats[ETHTOOL_A_HEADER_FLAGS + 1]; +extern const struct nla_policy ethnl_header_policy[ETHTOOL_A_HEADER_PHY_INDEX + 1]; +extern const struct nla_policy ethnl_header_policy_stats[ETHTOOL_A_HEADER_PHY_INDEX + 1]; extern const struct nla_policy ethnl_strset_get_policy[ETHTOOL_A_STRSET_COUNTS_ONLY + 1]; extern const struct nla_policy ethnl_linkinfo_get_policy[ETHTOOL_A_LINKINFO_HEADER + 1]; extern const struct nla_policy ethnl_linkinfo_set_policy[ETHTOOL_A_LINKINFO_TP_MDIX_CTRL + 1]; @@ -441,6 +444,7 @@ extern const struct nla_policy ethnl_plca_set_cfg_policy[ETHTOOL_A_PLCA_MAX + 1] extern const struct nla_policy ethnl_plca_get_status_policy[ETHTOOL_A_PLCA_HEADER + 1]; extern const struct nla_policy ethnl_mm_get_policy[ETHTOOL_A_MM_HEADER + 1]; extern const struct nla_policy ethnl_mm_set_policy[ETHTOOL_A_MM_MAX + 1]; +extern const struct nla_policy ethnl_phy_get_policy[ETHTOOL_A_PHY_HEADER + 1]; int ethnl_set_features(struct sk_buff *skb, struct genl_info *info); int ethnl_act_cable_test(struct sk_buff *skb, struct genl_info *info); @@ -448,6 +452,10 @@ int ethnl_act_cable_test_tdr(struct sk_buff *skb, struct genl_info *info); int ethnl_tunnel_info_doit(struct sk_buff *skb, struct genl_info *info); int ethnl_tunnel_info_start(struct netlink_callback *cb); int ethnl_tunnel_info_dumpit(struct sk_buff *skb, struct netlink_callback *cb); +int ethnl_phy_start(struct netlink_callback *cb); +int ethnl_phy_doit(struct sk_buff *skb, struct genl_info *info); +int ethnl_phy_dumpit(struct sk_buff *skb, struct netlink_callback *cb); +int ethnl_phy_done(struct netlink_callback *cb); extern const char stats_std_names[__ETHTOOL_STATS_CNT][ETH_GSTRING_LEN]; extern const char stats_eth_phy_names[__ETHTOOL_A_STATS_ETH_PHY_CNT][ETH_GSTRING_LEN]; diff --git a/net/ethtool/phy.c b/net/ethtool/phy.c new file mode 100644 index 000000000000..5add2840aaeb --- /dev/null +++ b/net/ethtool/phy.c @@ -0,0 +1,306 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2023 Bootlin + * + */ +#include "common.h" +#include "netlink.h" + +#include <linux/phy.h> +#include <linux/phy_link_topology.h> +#include <linux/sfp.h> + +struct phy_req_info { + struct ethnl_req_info base; + struct phy_device_node pdn; +}; + +#define PHY_REQINFO(__req_base) \ + container_of(__req_base, struct phy_req_info, base) + +const struct nla_policy ethnl_phy_get_policy[ETHTOOL_A_PHY_HEADER + 1] = { + [ETHTOOL_A_PHY_HEADER] = NLA_POLICY_NESTED(ethnl_header_policy), +}; + +/* Caller holds rtnl */ +static ssize_t +ethnl_phy_reply_size(const struct ethnl_req_info *req_base, + struct netlink_ext_ack *extack) +{ + struct phy_link_topology *topo; + struct phy_device_node *pdn; + struct phy_device *phydev; + unsigned long index; + size_t size; + + ASSERT_RTNL(); + + topo = &req_base->dev->link_topo; + + size = nla_total_size(0); + + xa_for_each(&topo->phys, index, pdn) { + phydev = pdn->phy; + + /* ETHTOOL_A_PHY_INDEX */ + size += nla_total_size(sizeof(u32)); + + /* ETHTOOL_A_DRVNAME */ + size += nla_total_size(strlen(phydev->drv->name) + 1); + + /* ETHTOOL_A_NAME */ + size += nla_total_size(strlen(dev_name(&phydev->mdio.dev)) + 1); + + /* ETHTOOL_A_PHY_UPSTREAM_TYPE */ + size += nla_total_size(sizeof(u8)); + + /* ETHTOOL_A_PHY_ID */ + size += nla_total_size(sizeof(u32)); + + if (phy_on_sfp(phydev)) { + const char *upstream_sfp_name = sfp_get_name(pdn->parent_sfp_bus); + + /* ETHTOOL_A_PHY_UPSTREAM_SFP_NAME */ + if (upstream_sfp_name) + size += nla_total_size(strlen(upstream_sfp_name) + 1); + + /* ETHTOOL_A_PHY_UPSTREAM_INDEX */ + size += nla_total_size(sizeof(u32)); + } + + /* ETHTOOL_A_PHY_DOWNSTREAM_SFP_NAME */ + if (phydev->sfp_bus) { + const char *sfp_name = sfp_get_name(phydev->sfp_bus); + + if (sfp_name) + size += nla_total_size(strlen(sfp_name) + 1); + } + } + + return size; +} + +static int +ethnl_phy_fill_reply(const struct ethnl_req_info *req_base, struct sk_buff *skb) +{ + struct phy_req_info *req_info = PHY_REQINFO(req_base); + struct phy_device_node *pdn = &req_info->pdn; + struct phy_device *phydev = pdn->phy; + enum phy_upstream ptype; + struct nlattr *nest; + + ptype = pdn->upstream_type; + + if (nla_put_u32(skb, ETHTOOL_A_PHY_INDEX, phydev->phyindex) || + nla_put_string(skb, ETHTOOL_A_PHY_DRVNAME, phydev->drv->name) || + nla_put_string(skb, ETHTOOL_A_PHY_NAME, dev_name(&phydev->mdio.dev)) || + nla_put_u8(skb, ETHTOOL_A_PHY_UPSTREAM_TYPE, ptype) || + nla_put_u32(skb, ETHTOOL_A_PHY_ID, phydev->phy_id)) + return -EMSGSIZE; + + if (ptype == PHY_UPSTREAM_PHY) { + struct phy_device *upstream = pdn->upstream.phydev; + const char *sfp_upstream_name; + + nest = nla_nest_start(skb, ETHTOOL_A_PHY_UPSTREAM); + if (!nest) + return -EMSGSIZE; + + /* Parent index */ + if (nla_put_u32(skb, ETHTOOL_A_PHY_UPSTREAM_INDEX, upstream->phyindex)) + return -EMSGSIZE; + + if (pdn->parent_sfp_bus) { + sfp_upstream_name = sfp_get_name(pdn->parent_sfp_bus); + if (sfp_upstream_name && nla_put_string(skb, + ETHTOOL_A_PHY_UPSTREAM_SFP_NAME, + sfp_upstream_name)) + return -EMSGSIZE; + } + + nla_nest_end(skb, nest); + } + + if (phydev->sfp_bus) { + const char *sfp_name = sfp_get_name(phydev->sfp_bus); + + if (sfp_name && + nla_put_string(skb, ETHTOOL_A_PHY_DOWNSTREAM_SFP_NAME, + sfp_name)) + return -EMSGSIZE; + } + + return 0; +} + +static int ethnl_phy_parse_request(struct ethnl_req_info *req_base, + struct nlattr **tb) +{ + struct phy_link_topology *topo = &req_base->dev->link_topo; + struct phy_req_info *req_info = PHY_REQINFO(req_base); + struct phy_device_node *pdn; + + if (!req_base->phydev) + return 0; + + pdn = xa_load(&topo->phys, req_base->phydev->phyindex); + memcpy(&req_info->pdn, pdn, sizeof(*pdn)); + + return 0; +} + +int ethnl_phy_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct phy_req_info req_info = {}; + struct nlattr **tb = info->attrs; + struct sk_buff *rskb; + void *reply_payload; + int reply_len; + int ret; + + ret = ethnl_parse_header_dev_get(&req_info.base, + tb[ETHTOOL_A_PHY_HEADER], + genl_info_net(info), info->extack, + true); + if (ret < 0) + return ret; + + rtnl_lock(); + + ret = ethnl_phy_parse_request(&req_info.base, tb); + if (ret < 0) + goto err_unlock_rtnl; + + /* No PHY, return early */ + if (!req_info.pdn.phy) + goto err_unlock_rtnl; + + ret = ethnl_phy_reply_size(&req_info.base, info->extack); + if (ret < 0) + goto err_unlock_rtnl; + reply_len = ret + ethnl_reply_header_size(); + + rskb = ethnl_reply_init(reply_len, req_info.base.dev, + ETHTOOL_MSG_PHY_GET_REPLY, + ETHTOOL_A_PHY_HEADER, + info, &reply_payload); + if (!rskb) { + ret = -ENOMEM; + goto err_unlock_rtnl; + } + + ret = ethnl_phy_fill_reply(&req_info.base, rskb); + if (ret) + goto err_free_msg; + + rtnl_unlock(); + ethnl_parse_header_dev_put(&req_info.base); + genlmsg_end(rskb, reply_payload); + + return genlmsg_reply(rskb, info); + +err_free_msg: + nlmsg_free(rskb); +err_unlock_rtnl: + rtnl_unlock(); + ethnl_parse_header_dev_put(&req_info.base); + return ret; +} + +struct ethnl_phy_dump_ctx { + struct phy_req_info *phy_req_info; +}; + +int ethnl_phy_start(struct netlink_callback *cb) +{ + const struct genl_dumpit_info *info = genl_dumpit_info(cb); + struct ethnl_phy_dump_ctx *ctx = (void *)cb->ctx; + struct nlattr **tb = info->info.attrs; + int ret; + + BUILD_BUG_ON(sizeof(*ctx) > sizeof(cb->ctx)); + + ctx->phy_req_info = kzalloc(sizeof(*ctx->phy_req_info), GFP_KERNEL); + if (!ctx->phy_req_info) + return -ENOMEM; + + ret = ethnl_parse_header_dev_get(&ctx->phy_req_info->base, + tb[ETHTOOL_A_PHY_HEADER], + sock_net(cb->skb->sk), cb->extack, + false); + return ret; +} + +int ethnl_phy_done(struct netlink_callback *cb) +{ + struct ethnl_phy_dump_ctx *ctx = (void *)cb->ctx; + + kfree(ctx->phy_req_info); + + return 0; +} + +static int ethnl_phy_dump_one_dev(struct sk_buff *skb, struct net_device *dev, + struct netlink_callback *cb) +{ + struct ethnl_phy_dump_ctx *ctx = (void *)cb->ctx; + struct phy_req_info *pri = ctx->phy_req_info; + struct phy_device_node *pdn; + unsigned long index = 1; + int ret = 0; + void *ehdr; + + pri->base.dev = dev; + + xa_for_each(&dev->link_topo.phys, index, pdn) { + ehdr = ethnl_dump_put(skb, cb, + ETHTOOL_MSG_PHY_GET_REPLY); + if (!ehdr) { + ret = -EMSGSIZE; + break; + } + + ret = ethnl_fill_reply_header(skb, dev, + ETHTOOL_A_PHY_HEADER); + if (ret < 0) { + genlmsg_cancel(skb, ehdr); + break; + } + + memcpy(&pri->pdn, pdn, sizeof(*pdn)); + ret = ethnl_phy_fill_reply(&pri->base, skb); + + genlmsg_end(skb, ehdr); + } + + return ret; +} + +int ethnl_phy_dumpit(struct sk_buff *skb, struct netlink_callback *cb) +{ + struct ethnl_phy_dump_ctx *ctx = (void *)cb->ctx; + struct net *net = sock_net(skb->sk); + unsigned long ifindex = 1; + struct net_device *dev; + int ret = 0; + + rtnl_lock(); + + if (ctx->phy_req_info->base.dev) { + ret = ethnl_phy_dump_one_dev(skb, ctx->phy_req_info->base.dev, cb); + ethnl_parse_header_dev_put(&ctx->phy_req_info->base); + ctx->phy_req_info->base.dev = NULL; + } else { + for_each_netdev_dump(net, dev, ifindex) { + ret = ethnl_phy_dump_one_dev(skb, dev, cb); + if (ret) + break; + } + } + rtnl_unlock(); + + if (ret == -EMSGSIZE && skb->len) + return skb->len; + return ret; +} + diff --git a/net/ethtool/plca.c b/net/ethtool/plca.c index b1e2e3b5027f..2b3e419f4dc2 100644 --- a/net/ethtool/plca.c +++ b/net/ethtool/plca.c @@ -61,7 +61,7 @@ static int plca_get_cfg_prepare_data(const struct ethnl_req_info *req_base, int ret; // check that the PHY device is available and connected - if (!dev->phydev) { + if (!req_base->phydev) { ret = -EOPNOTSUPP; goto out; } @@ -80,7 +80,7 @@ static int plca_get_cfg_prepare_data(const struct ethnl_req_info *req_base, memset(&data->plca_cfg, 0xff, sizeof_field(struct plca_reply_data, plca_cfg)); - ret = ops->get_plca_cfg(dev->phydev, &data->plca_cfg); + ret = ops->get_plca_cfg(req_base->phydev, &data->plca_cfg); ethnl_ops_complete(dev); out: @@ -141,7 +141,6 @@ const struct nla_policy ethnl_plca_set_cfg_policy[] = { static int ethnl_set_plca(struct ethnl_req_info *req_info, struct genl_info *info) { - struct net_device *dev = req_info->dev; const struct ethtool_phy_ops *ops; struct nlattr **tb = info->attrs; struct phy_plca_cfg plca_cfg; @@ -149,7 +148,7 @@ ethnl_set_plca(struct ethnl_req_info *req_info, struct genl_info *info) int ret; // check that the PHY device is available and connected - if (!dev->phydev) + if (!req_info->phydev) return -EOPNOTSUPP; ops = ethtool_phy_ops; @@ -168,7 +167,7 @@ ethnl_set_plca(struct ethnl_req_info *req_info, struct genl_info *info) if (!mod) return 0; - ret = ops->set_plca_cfg(dev->phydev, &plca_cfg, info->extack); + ret = ops->set_plca_cfg(req_info->phydev, &plca_cfg, info->extack); return ret < 0 ? ret : 1; } @@ -204,7 +203,7 @@ static int plca_get_status_prepare_data(const struct ethnl_req_info *req_base, int ret; // check that the PHY device is available and connected - if (!dev->phydev) { + if (!req_base->phydev) { ret = -EOPNOTSUPP; goto out; } @@ -223,7 +222,7 @@ static int plca_get_status_prepare_data(const struct ethnl_req_info *req_base, memset(&data->plca_st, 0xff, sizeof_field(struct plca_reply_data, plca_st)); - ret = ops->get_plca_status(dev->phydev, &data->plca_st); + ret = ops->get_plca_status(req_base->phydev, &data->plca_st); ethnl_ops_complete(dev); out: return ret; diff --git a/net/ethtool/pse-pd.c b/net/ethtool/pse-pd.c index cc478af77111..4a1c8d37bd3d 100644 --- a/net/ethtool/pse-pd.c +++ b/net/ethtool/pse-pd.c @@ -31,12 +31,10 @@ const struct nla_policy ethnl_pse_get_policy[ETHTOOL_A_PSE_HEADER + 1] = { [ETHTOOL_A_PSE_HEADER] = NLA_POLICY_NESTED(ethnl_header_policy), }; -static int pse_get_pse_attributes(struct net_device *dev, +static int pse_get_pse_attributes(struct phy_device *phydev, struct netlink_ext_ack *extack, struct pse_reply_data *data) { - struct phy_device *phydev = dev->phydev; - if (!phydev) { NL_SET_ERR_MSG(extack, "No PHY is attached"); return -EOPNOTSUPP; @@ -64,7 +62,7 @@ static int pse_prepare_data(const struct ethnl_req_info *req_base, if (ret < 0) return ret; - ret = pse_get_pse_attributes(dev, info->extack, data); + ret = pse_get_pse_attributes(req_base->phydev, info->extack, data); ethnl_ops_complete(dev); @@ -124,7 +122,6 @@ ethnl_set_pse_validate(struct ethnl_req_info *req_info, struct genl_info *info) static int ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info) { - struct net_device *dev = req_info->dev; struct pse_control_config config = {}; struct nlattr **tb = info->attrs; struct phy_device *phydev; @@ -132,7 +129,7 @@ ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info) /* this values are already validated by the ethnl_pse_set_policy */ config.admin_cotrol = nla_get_u32(tb[ETHTOOL_A_PODL_PSE_ADMIN_CONTROL]); - phydev = dev->phydev; + phydev = req_info->phydev; if (!phydev) { NL_SET_ERR_MSG(info->extack, "No PHY is attached"); return -EOPNOTSUPP; diff --git a/net/ethtool/strset.c b/net/ethtool/strset.c index c678b484a079..70c00631c51f 100644 --- a/net/ethtool/strset.c +++ b/net/ethtool/strset.c @@ -233,17 +233,18 @@ static void strset_cleanup_data(struct ethnl_reply_data *reply_base) } static int strset_prepare_set(struct strset_info *info, struct net_device *dev, - unsigned int id, bool counts_only) + struct phy_device *phydev, unsigned int id, + bool counts_only) { const struct ethtool_phy_ops *phy_ops = ethtool_phy_ops; const struct ethtool_ops *ops = dev->ethtool_ops; void *strings; int count, ret; - if (id == ETH_SS_PHY_STATS && dev->phydev && + if (id == ETH_SS_PHY_STATS && phydev && !ops->get_ethtool_phy_stats && phy_ops && phy_ops->get_sset_count) - ret = phy_ops->get_sset_count(dev->phydev); + ret = phy_ops->get_sset_count(phydev); else if (ops->get_sset_count && ops->get_strings) ret = ops->get_sset_count(dev, id); else @@ -258,10 +259,10 @@ static int strset_prepare_set(struct strset_info *info, struct net_device *dev, strings = kcalloc(count, ETH_GSTRING_LEN, GFP_KERNEL); if (!strings) return -ENOMEM; - if (id == ETH_SS_PHY_STATS && dev->phydev && + if (id == ETH_SS_PHY_STATS && phydev && !ops->get_ethtool_phy_stats && phy_ops && phy_ops->get_strings) - phy_ops->get_strings(dev->phydev, strings); + phy_ops->get_strings(phydev, strings); else ops->get_strings(dev, id, strings); info->strings = strings; @@ -305,8 +306,8 @@ static int strset_prepare_data(const struct ethnl_req_info *req_base, !data->sets[i].per_dev) continue; - ret = strset_prepare_set(&data->sets[i], dev, i, - req_info->counts_only); + ret = strset_prepare_set(&data->sets[i], dev, req_base->phydev, + i, req_info->counts_only); if (ret < 0) goto err_ops; } diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index bd325b029dd1..8e2eb1793685 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -159,8 +159,11 @@ static bool inet_use_bhash2_on_bind(const struct sock *sk) if (sk->sk_family == AF_INET6) { int addr_type = ipv6_addr_type(&sk->sk_v6_rcv_saddr); - return addr_type != IPV6_ADDR_ANY && - addr_type != IPV6_ADDR_MAPPED; + if (addr_type == IPV6_ADDR_ANY) + return false; + + if (addr_type != IPV6_ADDR_MAPPED) + return true; } #endif return sk->sk_rcv_saddr != htonl(INADDR_ANY); @@ -213,18 +216,9 @@ static bool inet_bhash2_conflict(const struct sock *sk, bool relax, bool reuseport_cb_ok, bool reuseport_ok) { - struct inet_timewait_sock *tw2; struct sock *sk2; - sk_for_each_bound_bhash2(sk2, &tb2->owners) { - if (__inet_bhash2_conflict(sk, sk2, sk_uid, relax, - reuseport_cb_ok, reuseport_ok)) - return true; - } - - twsk_for_each_bound_bhash2(tw2, &tb2->deathrow) { - sk2 = (struct sock *)tw2; - + sk_for_each_bound(sk2, &tb2->owners) { if (__inet_bhash2_conflict(sk, sk2, sk_uid, relax, reuseport_cb_ok, reuseport_ok)) return true; @@ -233,15 +227,20 @@ static bool inet_bhash2_conflict(const struct sock *sk, return false; } +#define sk_for_each_bound_bhash(__sk, __tb2, __tb) \ + hlist_for_each_entry(__tb2, &(__tb)->bhash2, bhash_node) \ + sk_for_each_bound(sk2, &(__tb2)->owners) + /* This should be called only when the tb and tb2 hashbuckets' locks are held */ static int inet_csk_bind_conflict(const struct sock *sk, const struct inet_bind_bucket *tb, const struct inet_bind2_bucket *tb2, /* may be null */ bool relax, bool reuseport_ok) { - bool reuseport_cb_ok; - struct sock_reuseport *reuseport_cb; kuid_t uid = sock_i_uid((struct sock *)sk); + struct sock_reuseport *reuseport_cb; + bool reuseport_cb_ok; + struct sock *sk2; rcu_read_lock(); reuseport_cb = rcu_dereference(sk->sk_reuseport_cb); @@ -249,32 +248,29 @@ static int inet_csk_bind_conflict(const struct sock *sk, reuseport_cb_ok = !reuseport_cb || READ_ONCE(reuseport_cb->num_closed_socks); rcu_read_unlock(); - /* - * Unlike other sk lookup places we do not check + /* Conflicts with an existing IPV6_ADDR_ANY (if ipv6) or INADDR_ANY (if + * ipv4) should have been checked already. We need to do these two + * checks separately because their spinlocks have to be acquired/released + * independently of each other, to prevent possible deadlocks + */ + if (inet_use_bhash2_on_bind(sk)) + return tb2 && inet_bhash2_conflict(sk, tb2, uid, relax, + reuseport_cb_ok, reuseport_ok); + + /* Unlike other sk lookup places we do not check * for sk_net here, since _all_ the socks listed * in tb->owners and tb2->owners list belong * to the same net - the one this bucket belongs to. */ + sk_for_each_bound_bhash(sk2, tb2, tb) { + if (!inet_bind_conflict(sk, sk2, uid, relax, reuseport_cb_ok, reuseport_ok)) + continue; - if (!inet_use_bhash2_on_bind(sk)) { - struct sock *sk2; - - sk_for_each_bound(sk2, &tb->owners) - if (inet_bind_conflict(sk, sk2, uid, relax, - reuseport_cb_ok, reuseport_ok) && - inet_rcv_saddr_equal(sk, sk2, true)) - return true; - - return false; + if (inet_rcv_saddr_equal(sk, sk2, true)) + return true; } - /* Conflicts with an existing IPV6_ADDR_ANY (if ipv6) or INADDR_ANY (if - * ipv4) should have been checked already. We need to do these two - * checks separately because their spinlocks have to be acquired/released - * independently of each other, to prevent possible deadlocks - */ - return tb2 && inet_bhash2_conflict(sk, tb2, uid, relax, reuseport_cb_ok, - reuseport_ok); + return false; } /* Determine if there is a bind conflict with an existing IPV6_ADDR_ANY (if ipv6) or @@ -457,7 +453,7 @@ void inet_csk_update_fastreuse(struct inet_bind_bucket *tb, kuid_t uid = sock_i_uid(sk); bool reuse = sk->sk_reuse && sk->sk_state != TCP_LISTEN; - if (hlist_empty(&tb->owners)) { + if (hlist_empty(&tb->bhash2)) { tb->fastreuse = reuse; if (sk->sk_reuseport) { tb->fastreuseport = FASTREUSEPORT_ANY; @@ -549,7 +545,7 @@ int inet_csk_get_port(struct sock *sk, unsigned short snum) } if (!found_port) { - if (!hlist_empty(&tb->owners)) { + if (!hlist_empty(&tb->bhash2)) { if (sk->sk_reuse == SK_FORCE_REUSE || (tb->fastreuse > 0 && reuse) || sk_reuseport_match(tb, sk)) @@ -569,7 +565,7 @@ int inet_csk_get_port(struct sock *sk, unsigned short snum) if (!tb2) { tb2 = inet_bind2_bucket_create(hinfo->bind2_bucket_cachep, - net, head2, port, l3mdev, sk); + net, head2, tb, sk); if (!tb2) goto fail_unlock; bhash2_created = true; @@ -591,11 +587,10 @@ success: fail_unlock: if (ret) { + if (bhash2_created) + inet_bind2_bucket_destroy(hinfo->bind2_bucket_cachep, tb2); if (bhash_created) inet_bind_bucket_destroy(hinfo->bind_bucket_cachep, tb); - if (bhash2_created) - inet_bind2_bucket_destroy(hinfo->bind2_bucket_cachep, - tb2); } if (head2_lock_acquired) spin_unlock(&head2->lock); diff --git a/net/ipv4/inet_diag.c b/net/ipv4/inet_diag.c index 46b13962ad02..8e6b6aa0579e 100644 --- a/net/ipv4/inet_diag.c +++ b/net/ipv4/inet_diag.c @@ -1104,7 +1104,7 @@ resume_bind_walk: if (!net_eq(ib2_net(tb2), net)) continue; - sk_for_each_bound_bhash2(sk, &tb2->owners) { + sk_for_each_bound(sk, &tb2->owners) { struct inet_sock *inet = inet_sk(sk); if (num < s_num) diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 9ff201bc4e6d..93e9193df544 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -76,7 +76,7 @@ struct inet_bind_bucket *inet_bind_bucket_create(struct kmem_cache *cachep, tb->port = snum; tb->fastreuse = 0; tb->fastreuseport = 0; - INIT_HLIST_HEAD(&tb->owners); + INIT_HLIST_HEAD(&tb->bhash2); hlist_add_head(&tb->node, &head->chain); } return tb; @@ -87,7 +87,7 @@ struct inet_bind_bucket *inet_bind_bucket_create(struct kmem_cache *cachep, */ void inet_bind_bucket_destroy(struct kmem_cache *cachep, struct inet_bind_bucket *tb) { - if (hlist_empty(&tb->owners)) { + if (hlist_empty(&tb->bhash2)) { __hlist_del(&tb->node); kmem_cache_free(cachep, tb); } @@ -100,47 +100,52 @@ bool inet_bind_bucket_match(const struct inet_bind_bucket *tb, const struct net tb->l3mdev == l3mdev; } -static void inet_bind2_bucket_init(struct inet_bind2_bucket *tb, +static void inet_bind2_bucket_init(struct inet_bind2_bucket *tb2, struct net *net, struct inet_bind_hashbucket *head, - unsigned short port, int l3mdev, + struct inet_bind_bucket *tb, const struct sock *sk) { - write_pnet(&tb->ib_net, net); - tb->l3mdev = l3mdev; - tb->port = port; + write_pnet(&tb2->ib_net, net); + tb2->l3mdev = tb->l3mdev; + tb2->port = tb->port; #if IS_ENABLED(CONFIG_IPV6) - tb->family = sk->sk_family; - if (sk->sk_family == AF_INET6) - tb->v6_rcv_saddr = sk->sk_v6_rcv_saddr; - else + BUILD_BUG_ON(USHRT_MAX < (IPV6_ADDR_ANY | IPV6_ADDR_MAPPED)); + if (sk->sk_family == AF_INET6) { + tb2->addr_type = ipv6_addr_type(&sk->sk_v6_rcv_saddr); + tb2->v6_rcv_saddr = sk->sk_v6_rcv_saddr; + } else { + tb2->addr_type = IPV6_ADDR_MAPPED; + ipv6_addr_set_v4mapped(sk->sk_rcv_saddr, &tb2->v6_rcv_saddr); + } +#else + tb2->rcv_saddr = sk->sk_rcv_saddr; #endif - tb->rcv_saddr = sk->sk_rcv_saddr; - INIT_HLIST_HEAD(&tb->owners); - INIT_HLIST_HEAD(&tb->deathrow); - hlist_add_head(&tb->node, &head->chain); + INIT_HLIST_HEAD(&tb2->owners); + hlist_add_head(&tb2->node, &head->chain); + hlist_add_head(&tb2->bhash_node, &tb->bhash2); } struct inet_bind2_bucket *inet_bind2_bucket_create(struct kmem_cache *cachep, struct net *net, struct inet_bind_hashbucket *head, - unsigned short port, - int l3mdev, + struct inet_bind_bucket *tb, const struct sock *sk) { - struct inet_bind2_bucket *tb = kmem_cache_alloc(cachep, GFP_ATOMIC); + struct inet_bind2_bucket *tb2 = kmem_cache_alloc(cachep, GFP_ATOMIC); - if (tb) - inet_bind2_bucket_init(tb, net, head, port, l3mdev, sk); + if (tb2) + inet_bind2_bucket_init(tb2, net, head, tb, sk); - return tb; + return tb2; } /* Caller must hold hashbucket lock for this tb with local BH disabled */ void inet_bind2_bucket_destroy(struct kmem_cache *cachep, struct inet_bind2_bucket *tb) { - if (hlist_empty(&tb->owners) && hlist_empty(&tb->deathrow)) { + if (hlist_empty(&tb->owners)) { __hlist_del(&tb->node); + __hlist_del(&tb->bhash_node); kmem_cache_free(cachep, tb); } } @@ -149,18 +154,11 @@ static bool inet_bind2_bucket_addr_match(const struct inet_bind2_bucket *tb2, const struct sock *sk) { #if IS_ENABLED(CONFIG_IPV6) - if (sk->sk_family != tb2->family) { - if (sk->sk_family == AF_INET) - return ipv6_addr_v4mapped(&tb2->v6_rcv_saddr) && - tb2->v6_rcv_saddr.s6_addr32[3] == sk->sk_rcv_saddr; - - return ipv6_addr_v4mapped(&sk->sk_v6_rcv_saddr) && - sk->sk_v6_rcv_saddr.s6_addr32[3] == tb2->rcv_saddr; - } - if (sk->sk_family == AF_INET6) - return ipv6_addr_equal(&tb2->v6_rcv_saddr, - &sk->sk_v6_rcv_saddr); + return ipv6_addr_equal(&tb2->v6_rcv_saddr, &sk->sk_v6_rcv_saddr); + + if (tb2->addr_type != IPV6_ADDR_MAPPED) + return false; #endif return tb2->rcv_saddr == sk->sk_rcv_saddr; } @@ -169,10 +167,9 @@ void inet_bind_hash(struct sock *sk, struct inet_bind_bucket *tb, struct inet_bind2_bucket *tb2, unsigned short port) { inet_sk(sk)->inet_num = port; - sk_add_bind_node(sk, &tb->owners); inet_csk(sk)->icsk_bind_hash = tb; - sk_add_bind2_node(sk, &tb2->owners); inet_csk(sk)->icsk_bind2_hash = tb2; + sk_add_bind_node(sk, &tb2->owners); } /* @@ -192,21 +189,20 @@ static void __inet_put_port(struct sock *sk) spin_lock(&head->lock); tb = inet_csk(sk)->icsk_bind_hash; - __sk_del_bind_node(sk); inet_csk(sk)->icsk_bind_hash = NULL; inet_sk(sk)->inet_num = 0; - inet_bind_bucket_destroy(hashinfo->bind_bucket_cachep, tb); spin_lock(&head2->lock); if (inet_csk(sk)->icsk_bind2_hash) { struct inet_bind2_bucket *tb2 = inet_csk(sk)->icsk_bind2_hash; - __sk_del_bind2_node(sk); + __sk_del_bind_node(sk); inet_csk(sk)->icsk_bind2_hash = NULL; inet_bind2_bucket_destroy(hashinfo->bind2_bucket_cachep, tb2); } spin_unlock(&head2->lock); + inet_bind_bucket_destroy(hashinfo->bind_bucket_cachep, tb); spin_unlock(&head->lock); } @@ -275,8 +271,7 @@ bhash2_find: tb2 = inet_bind2_bucket_find(head2, net, port, l3mdev, child); if (!tb2) { tb2 = inet_bind2_bucket_create(table->bind2_bucket_cachep, - net, head2, port, - l3mdev, child); + net, head2, tb, child); if (!tb2) goto error; } @@ -836,16 +831,15 @@ bool inet_bind2_bucket_match_addr_any(const struct inet_bind2_bucket *tb, const return false; #if IS_ENABLED(CONFIG_IPV6) - if (sk->sk_family != tb->family) { - if (sk->sk_family == AF_INET) - return ipv6_addr_any(&tb->v6_rcv_saddr) || - ipv6_addr_v4mapped_any(&tb->v6_rcv_saddr); + if (tb->addr_type == IPV6_ADDR_ANY) + return true; + if (tb->addr_type != IPV6_ADDR_MAPPED) return false; - } - if (sk->sk_family == AF_INET6) - return ipv6_addr_any(&tb->v6_rcv_saddr); + if (sk->sk_family == AF_INET6 && + !ipv6_addr_v4mapped(&sk->sk_v6_rcv_saddr)) + return false; #endif return tb->rcv_saddr == 0; } @@ -942,7 +936,7 @@ static int __inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family, spin_lock_bh(&head->lock); spin_lock(&head2->lock); - __sk_del_bind2_node(sk); + __sk_del_bind_node(sk); inet_bind2_bucket_destroy(hinfo->bind2_bucket_cachep, inet_csk(sk)->icsk_bind2_hash); spin_unlock(&head2->lock); @@ -957,10 +951,10 @@ static int __inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family, tb2 = inet_bind2_bucket_find(head2, net, port, l3mdev, sk); if (!tb2) { tb2 = new_tb2; - inet_bind2_bucket_init(tb2, net, head2, port, l3mdev, sk); + inet_bind2_bucket_init(tb2, net, head2, inet_csk(sk)->icsk_bind_hash, sk); } - sk_add_bind2_node(sk, &tb2->owners); inet_csk(sk)->icsk_bind2_hash = tb2; + sk_add_bind_node(sk, &tb2->owners); spin_unlock(&head2->lock); spin_unlock_bh(&head->lock); @@ -1064,7 +1058,7 @@ other_parity_scan: if (tb->fastreuse >= 0 || tb->fastreuseport >= 0) goto next_port; - WARN_ON(hlist_empty(&tb->owners)); + WARN_ON(hlist_empty(&tb->bhash2)); if (!check_established(death_row, sk, port, &tw)) goto ok; @@ -1104,7 +1098,7 @@ ok: tb2 = inet_bind2_bucket_find(head2, net, port, l3mdev, sk); if (!tb2) { tb2 = inet_bind2_bucket_create(hinfo->bind2_bucket_cachep, net, - head2, port, l3mdev, sk); + head2, tb, sk); if (!tb2) goto error; } diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c index dd37a5bf6881..5befa4de5b24 100644 --- a/net/ipv4/inet_timewait_sock.c +++ b/net/ipv4/inet_timewait_sock.c @@ -35,13 +35,11 @@ void inet_twsk_bind_unhash(struct inet_timewait_sock *tw, if (!tb) return; - __hlist_del(&tw->tw_bind_node); + __sk_del_bind_node((struct sock *)tw); tw->tw_tb = NULL; - inet_bind_bucket_destroy(hashinfo->bind_bucket_cachep, tb); - - __hlist_del(&tw->tw_bind2_node); tw->tw_tb2 = NULL; inet_bind2_bucket_destroy(hashinfo->bind2_bucket_cachep, tb2); + inet_bind_bucket_destroy(hashinfo->bind_bucket_cachep, tb); __sock_put((struct sock *)tw); } @@ -94,18 +92,6 @@ static void inet_twsk_add_node_rcu(struct inet_timewait_sock *tw, hlist_nulls_add_head_rcu(&tw->tw_node, list); } -static void inet_twsk_add_bind_node(struct inet_timewait_sock *tw, - struct hlist_head *list) -{ - hlist_add_head(&tw->tw_bind_node, list); -} - -static void inet_twsk_add_bind2_node(struct inet_timewait_sock *tw, - struct hlist_head *list) -{ - hlist_add_head(&tw->tw_bind2_node, list); -} - /* * Enter the time wait state. This is called with locally disabled BH. * Essentially we whip up a timewait bucket, copy the relevant info into it @@ -133,11 +119,10 @@ void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk, tw->tw_tb = icsk->icsk_bind_hash; WARN_ON(!icsk->icsk_bind_hash); - inet_twsk_add_bind_node(tw, &tw->tw_tb->owners); tw->tw_tb2 = icsk->icsk_bind2_hash; WARN_ON(!icsk->icsk_bind2_hash); - inet_twsk_add_bind2_node(tw, &tw->tw_tb2->deathrow); + sk_add_bind_node((struct sock *)tw, &tw->tw_tb2->owners); spin_unlock(&bhead2->lock); spin_unlock(&bhead->lock); diff --git a/net/iucv/iucv.c b/net/iucv/iucv.c index 0ed6e34d6edd..6334f64f04d5 100644 --- a/net/iucv/iucv.c +++ b/net/iucv/iucv.c @@ -67,7 +67,7 @@ static int iucv_bus_match(struct device *dev, struct device_driver *drv) return 0; } -struct bus_type iucv_bus = { +const struct bus_type iucv_bus = { .name = "iucv", .match = iucv_bus_match, }; diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h index e9219f927875..eb482fb8c3af 100644 --- a/net/mac80211/driver-ops.h +++ b/net/mac80211/driver-ops.h @@ -1666,6 +1666,26 @@ static inline int drv_net_setup_tc(struct ieee80211_local *local, return ret; } +static inline bool drv_can_activate_links(struct ieee80211_local *local, + struct ieee80211_sub_if_data *sdata, + u16 active_links) +{ + bool ret = true; + + lockdep_assert_wiphy(local->hw.wiphy); + + if (!check_sdata_in_driver(sdata)) + return false; + + trace_drv_can_activate_links(local, sdata, active_links); + if (local->ops->can_activate_links) + ret = local->ops->can_activate_links(&local->hw, &sdata->vif, + active_links); + trace_drv_return_bool(local, ret); + + return ret; +} + int drv_change_vif_links(struct ieee80211_local *local, struct ieee80211_sub_if_data *sdata, u16 old_links, u16 new_links, diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 29312f6638a1..0ed82cc263f2 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -439,6 +439,7 @@ struct ieee80211_mgd_assoc_data { bool need_beacon; bool synced; bool timeout_started; + bool comeback; /* whether the AP has requested association comeback */ bool s1g; unsigned int assoc_link_id; @@ -1775,10 +1776,7 @@ static inline bool txq_has_queue(struct ieee80211_txq *txq) static inline bool ieee80211_have_rx_timestamp(struct ieee80211_rx_status *status) { - WARN_ON_ONCE(status->flag & RX_FLAG_MACTIME_START && - status->flag & RX_FLAG_MACTIME_END); - return !!(status->flag & (RX_FLAG_MACTIME_START | RX_FLAG_MACTIME_END | - RX_FLAG_MACTIME_PLCP_START)); + return status->flag & RX_FLAG_MACTIME; } void ieee80211_vif_inc_num_mcast(struct ieee80211_sub_if_data *sdata); diff --git a/net/mac80211/link.c b/net/mac80211/link.c index bf7bd880d062..d4f86955afa6 100644 --- a/net/mac80211/link.c +++ b/net/mac80211/link.c @@ -444,6 +444,9 @@ int ieee80211_set_active_links(struct ieee80211_vif *vif, u16 active_links) lockdep_assert_wiphy(local->hw.wiphy); + if (!drv_can_activate_links(local, sdata, active_links)) + return -EINVAL; + old_active = sdata->vif.active_links; if (old_active & active_links) { /* diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index 40a4fbfff530..967282baf0e2 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -43,6 +43,9 @@ #define IEEE80211_ASSOC_TIMEOUT_SHORT (HZ / 10) #define IEEE80211_ASSOC_MAX_TRIES 3 +#define IEEE80211_ADV_TTLM_SAFETY_BUFFER_MS msecs_to_jiffies(100) +#define IEEE80211_ADV_TTLM_ST_UNDERFLOW 0xff00 + static int max_nullfunc_tries = 2; module_param(max_nullfunc_tries, int, 0644); MODULE_PARM_DESC(max_nullfunc_tries, @@ -598,6 +601,7 @@ static int ieee80211_config_bw(struct ieee80211_link_data *link, return ret; } + cfg80211_schedule_channels_check(&sdata->wdev); return 0; } @@ -5381,6 +5385,7 @@ static void ieee80211_rx_mgmt_assoc_resp(struct ieee80211_sub_if_data *sdata, assoc_data->ap_addr, tu, ms); assoc_data->timeout = jiffies + msecs_to_jiffies(ms); assoc_data->timeout_started = true; + assoc_data->comeback = true; if (ms > IEEE80211_ASSOC_TIMEOUT) run_again(sdata, assoc_data->timeout); goto notify_driver; @@ -5964,6 +5969,13 @@ ieee80211_parse_adv_t2l(struct ieee80211_sub_if_data *sdata, pos++; ttlm_info->switch_time = get_unaligned_le16(pos); + + /* Since ttlm_info->switch_time == 0 means no switch time, bump it + * by 1. + */ + if (!ttlm_info->switch_time) + ttlm_info->switch_time = 1; + pos += 2; if (control & IEEE80211_TTLM_CONTROL_EXPECTED_DUR_PRESENT) { @@ -6058,25 +6070,46 @@ static void ieee80211_process_adv_ttlm(struct ieee80211_sub_if_data *sdata, } if (ttlm_info.switch_time) { - u32 st_us, delay = 0; - u32 ts_l26 = beacon_ts & GENMASK(25, 0); + u16 beacon_ts_tu, st_tu, delay; + u32 delay_jiffies; + u64 mask; /* The t2l map switch time is indicated with a partial - * TSF value, convert it to TSF and calc the delay - * to the start time. + * TSF value (bits 10 to 25), get the partial beacon TS + * as well, and calc the delay to the start time. + */ + mask = GENMASK_ULL(25, 10); + beacon_ts_tu = (beacon_ts & mask) >> 10; + st_tu = ttlm_info.switch_time; + delay = st_tu - beacon_ts_tu; + + /* + * If the switch time is far in the future, then it + * could also be the previous switch still being + * announced. + * We can simply ignore it for now, if it is a future + * switch the AP will continue to announce it anyway. */ - st_us = ieee80211_tu_to_usec(ttlm_info.switch_time); - if (st_us > ts_l26) - delay = st_us - ts_l26; + if (delay > IEEE80211_ADV_TTLM_ST_UNDERFLOW) + return; + + delay_jiffies = TU_TO_JIFFIES(delay); + + /* Link switching can take time, so schedule it + * 100ms before to be ready on time + */ + if (delay_jiffies > IEEE80211_ADV_TTLM_SAFETY_BUFFER_MS) + delay_jiffies -= + IEEE80211_ADV_TTLM_SAFETY_BUFFER_MS; else - continue; + delay_jiffies = 0; sdata->u.mgd.ttlm_info = ttlm_info; wiphy_delayed_work_cancel(sdata->local->hw.wiphy, &sdata->u.mgd.ttlm_work); wiphy_delayed_work_queue(sdata->local->hw.wiphy, &sdata->u.mgd.ttlm_work, - usecs_to_jiffies(delay)); + delay_jiffies); return; } } @@ -6720,8 +6753,18 @@ void ieee80211_sta_work(struct ieee80211_sub_if_data *sdata) } ifmgd->auth_data->timeout_started = true; } else if (ifmgd->assoc_data && + !ifmgd->assoc_data->comeback && (ieee80211_is_assoc_req(fc) || ieee80211_is_reassoc_req(fc))) { + /* + * Update association timeout based on the TX status + * for the (Re)Association Request frame. Skip this if + * we have already processed a (Re)Association Response + * frame that indicated need for association comeback + * at a specific time in the future. This could happen + * if the TX status information is delayed enough for + * the response to be received and processed first. + */ if (status_acked) { ifmgd->assoc_data->timeout = jiffies + IEEE80211_ASSOC_TIMEOUT_SHORT; diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c index bbfdcb0ade72..a57c8272c1dc 100644 --- a/net/mac80211/rx.c +++ b/net/mac80211/rx.c @@ -566,7 +566,8 @@ ieee80211_add_rx_radiotap_header(struct ieee80211_local *local, if (local->hw.radiotap_timestamp.units_pos >= 0) { u16 accuracy = 0; - u8 flags = IEEE80211_RADIOTAP_TIMESTAMP_FLAG_32BIT; + u8 flags; + u64 ts; rthdr->it_present |= cpu_to_le32(BIT(IEEE80211_RADIOTAP_TIMESTAMP)); @@ -575,7 +576,15 @@ ieee80211_add_rx_radiotap_header(struct ieee80211_local *local, while ((pos - (u8 *)rthdr) & 7) pos++; - put_unaligned_le64(status->device_timestamp, pos); + if (status->flag & RX_FLAG_MACTIME_IS_RTAP_TS64) { + flags = IEEE80211_RADIOTAP_TIMESTAMP_FLAG_64BIT; + ts = status->mactime; + } else { + flags = IEEE80211_RADIOTAP_TIMESTAMP_FLAG_32BIT; + ts = status->device_timestamp; + } + + put_unaligned_le64(ts, pos); pos += sizeof(u64); if (local->hw.radiotap_timestamp.accuracy >= 0) { diff --git a/net/mac80211/trace.h b/net/mac80211/trace.h index 032718d5b298..06835ed4c44f 100644 --- a/net/mac80211/trace.h +++ b/net/mac80211/trace.h @@ -2512,6 +2512,31 @@ TRACE_EVENT(drv_net_setup_tc, ) ); +TRACE_EVENT(drv_can_activate_links, + TP_PROTO(struct ieee80211_local *local, + struct ieee80211_sub_if_data *sdata, + u16 active_links), + + TP_ARGS(local, sdata, active_links), + + TP_STRUCT__entry( + LOCAL_ENTRY + VIF_ENTRY + __field(u16, active_links) + ), + + TP_fast_assign( + LOCAL_ASSIGN; + VIF_ASSIGN; + __entry->active_links = active_links; + ), + + TP_printk( + LOCAL_PR_FMT VIF_PR_FMT " requested active_links:0x%04x\n", + LOCAL_PR_ARG, VIF_PR_ARG, __entry->active_links + ) +); + TRACE_EVENT(drv_change_vif_links, TP_PROTO(struct ieee80211_local *local, struct ieee80211_sub_if_data *sdata, diff --git a/net/mac80211/util.c b/net/mac80211/util.c index ed680120d5a7..643c54855be6 100644 --- a/net/mac80211/util.c +++ b/net/mac80211/util.c @@ -4176,6 +4176,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, unsigned int mpdu_offset) { u64 ts = status->mactime; + bool mactime_plcp_start; struct rate_info ri; u16 rate; u8 n_ltf; @@ -4183,6 +4184,9 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, if (WARN_ON(!ieee80211_have_rx_timestamp(status))) return 0; + mactime_plcp_start = (status->flag & RX_FLAG_MACTIME) == + RX_FLAG_MACTIME_PLCP_START; + memset(&ri, 0, sizeof(ri)); ri.bw = status->bw; @@ -4197,7 +4201,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, if (status->enc_flags & RX_ENC_FLAG_SHORT_GI) ri.flags |= RATE_INFO_FLAGS_SHORT_GI; /* TODO/FIXME: is this right? handle other PPDUs */ - if (status->flag & RX_FLAG_MACTIME_PLCP_START) { + if (mactime_plcp_start) { mpdu_offset += 2; ts += 36; } @@ -4214,7 +4218,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, * See P802.11ax_D6.0, section 27.3.4 for * VHT PPDU format. */ - if (status->flag & RX_FLAG_MACTIME_PLCP_START) { + if (mactime_plcp_start) { mpdu_offset += 2; ts += 36; @@ -4238,7 +4242,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, * See P802.11REVmd_D3.0, section 19.3.2 for * HT PPDU format. */ - if (status->flag & RX_FLAG_MACTIME_PLCP_START) { + if (mactime_plcp_start) { mpdu_offset += 2; if (status->enc_flags & RX_ENC_FLAG_HT_GF) ts += 24; @@ -4266,7 +4270,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, * See P802.11REVmd_D3.0, section 21.3.2 for * VHT PPDU format. */ - if (status->flag & RX_FLAG_MACTIME_PLCP_START) { + if (mactime_plcp_start) { mpdu_offset += 2; ts += 36; @@ -4288,7 +4292,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, sband = local->hw.wiphy->bands[status->band]; ri.legacy = sband->bitrates[status->rate_idx].bitrate; - if (status->flag & RX_FLAG_MACTIME_PLCP_START) { + if (mactime_plcp_start) { if (status->band == NL80211_BAND_5GHZ) { ts += 20; mpdu_offset += 2; @@ -4310,7 +4314,7 @@ u64 ieee80211_calculate_rx_timestamp(struct ieee80211_local *local, return 0; /* rewind from end of MPDU */ - if (status->flag & RX_FLAG_MACTIME_END) + if ((status->flag & RX_FLAG_MACTIME) == RX_FLAG_MACTIME_END) ts -= mpdu_len * 8 * 10 / rate; ts += mpdu_offset * 8 * 10 / rate; diff --git a/net/mptcp/mib.c b/net/mptcp/mib.c index a0990c365a2e..c30405e76833 100644 --- a/net/mptcp/mib.c +++ b/net/mptcp/mib.c @@ -66,6 +66,7 @@ static const struct snmp_mib mptcp_snmp_list[] = { SNMP_MIB_ITEM("RcvWndShared", MPTCP_MIB_RCVWNDSHARED), SNMP_MIB_ITEM("RcvWndConflictUpdate", MPTCP_MIB_RCVWNDCONFLICTUPDATE), SNMP_MIB_ITEM("RcvWndConflict", MPTCP_MIB_RCVWNDCONFLICT), + SNMP_MIB_ITEM("MPCurrEstab", MPTCP_MIB_CURRESTAB), SNMP_MIB_SENTINEL }; diff --git a/net/mptcp/mib.h b/net/mptcp/mib.h index cae71d947252..dd7fd1f246b5 100644 --- a/net/mptcp/mib.h +++ b/net/mptcp/mib.h @@ -65,6 +65,7 @@ enum linux_mptcp_mib_field { * conflict with another subflow while updating msk rcv wnd */ MPTCP_MIB_RCVWNDCONFLICT, /* Conflict with while updating msk rcv wnd */ + MPTCP_MIB_CURRESTAB, /* Current established MPTCP connections */ __MPTCP_MIB_MAX }; @@ -95,4 +96,11 @@ static inline void __MPTCP_INC_STATS(struct net *net, __SNMP_INC_STATS(net->mib.mptcp_statistics, field); } +static inline void MPTCP_DEC_STATS(struct net *net, + enum linux_mptcp_mib_field field) +{ + if (likely(net->mib.mptcp_statistics)) + SNMP_DEC_STATS(net->mib.mptcp_statistics, field); +} + bool mptcp_mib_alloc(struct net *net); diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c index bf4d96f6f99a..287a60381eae 100644 --- a/net/mptcp/pm_netlink.c +++ b/net/mptcp/pm_netlink.c @@ -1048,6 +1048,11 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk, if (err) return err; + /* We don't use mptcp_set_state() here because it needs to be called + * under the msk socket lock. For the moment, that will not bring + * anything more than only calling inet_sk_state_store(), because the + * old status is known (TCP_CLOSE). + */ inet_sk_state_store(newsk, TCP_LISTEN); lock_sock(ssk); err = __inet_listen_sk(ssk, backlog); @@ -1100,7 +1105,7 @@ int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct mptcp_addr_info *skc static const struct genl_multicast_group mptcp_pm_mcgrps[] = { [MPTCP_PM_CMD_GRP_OFFSET] = { .name = MPTCP_PM_CMD_GRP_NAME, }, [MPTCP_PM_EV_GRP_OFFSET] = { .name = MPTCP_PM_EV_GRP_NAME, - .flags = GENL_UNS_ADMIN_PERM, + .flags = GENL_MCAST_CAP_NET_ADMIN, }, }; diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 5cd5c3f535a8..3ed4709a7509 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -55,28 +55,14 @@ static u64 mptcp_wnd_end(const struct mptcp_sock *msk) return READ_ONCE(msk->wnd_end); } -static bool mptcp_is_tcpsk(struct sock *sk) +static const struct proto_ops *mptcp_fallback_tcp_ops(const struct sock *sk) { - struct socket *sock = sk->sk_socket; - - if (unlikely(sk->sk_prot == &tcp_prot)) { - /* we are being invoked after mptcp_accept() has - * accepted a non-mp-capable flow: sk is a tcp_sk, - * not an mptcp one. - * - * Hand the socket over to tcp so all further socket ops - * bypass mptcp. - */ - WRITE_ONCE(sock->ops, &inet_stream_ops); - return true; #if IS_ENABLED(CONFIG_MPTCP_IPV6) - } else if (unlikely(sk->sk_prot == &tcpv6_prot)) { - WRITE_ONCE(sock->ops, &inet6_stream_ops); - return true; + if (sk->sk_prot == &tcpv6_prot) + return &inet6_stream_ops; #endif - } - - return false; + WARN_ON_ONCE(sk->sk_prot != &tcp_prot); + return &inet_stream_ops; } static int __mptcp_socket_create(struct mptcp_sock *msk) @@ -443,11 +429,11 @@ static void mptcp_check_data_fin_ack(struct sock *sk) switch (sk->sk_state) { case TCP_FIN_WAIT1: - inet_sk_state_store(sk, TCP_FIN_WAIT2); + mptcp_set_state(sk, TCP_FIN_WAIT2); break; case TCP_CLOSING: case TCP_LAST_ACK: - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); break; } @@ -608,13 +594,13 @@ static bool mptcp_check_data_fin(struct sock *sk) switch (sk->sk_state) { case TCP_ESTABLISHED: - inet_sk_state_store(sk, TCP_CLOSE_WAIT); + mptcp_set_state(sk, TCP_CLOSE_WAIT); break; case TCP_FIN_WAIT1: - inet_sk_state_store(sk, TCP_CLOSING); + mptcp_set_state(sk, TCP_CLOSING); break; case TCP_FIN_WAIT2: - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); break; default: /* Other states not expected */ @@ -789,7 +775,7 @@ static bool __mptcp_subflow_error_report(struct sock *sk, struct sock *ssk) */ ssk_state = inet_sk_state_load(ssk); if (ssk_state == TCP_CLOSE && !sock_flag(sk, SOCK_DEAD)) - inet_sk_state_store(sk, ssk_state); + mptcp_set_state(sk, ssk_state); WRITE_ONCE(sk->sk_err, -err); /* This barrier is coupled with smp_rmb() in mptcp_poll() */ @@ -2477,7 +2463,7 @@ out: inet_sk_state_load(msk->first) == TCP_CLOSE) { if (sk->sk_state != TCP_ESTABLISHED || msk->in_accept_queue || sock_flag(sk, SOCK_DEAD)) { - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); mptcp_close_wake_up(sk); } else { mptcp_start_tout_timer(sk); @@ -2572,7 +2558,7 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk) WRITE_ONCE(sk->sk_err, ECONNRESET); } - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); WRITE_ONCE(sk->sk_shutdown, SHUTDOWN_MASK); smp_mb__before_atomic(); /* SHUTDOWN must be visible first */ set_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags); @@ -2707,7 +2693,7 @@ static void mptcp_do_fastclose(struct sock *sk) struct mptcp_subflow_context *subflow, *tmp; struct mptcp_sock *msk = mptcp_sk(sk); - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); mptcp_for_each_subflow_safe(msk, subflow, tmp) __mptcp_close_ssk(sk, mptcp_subflow_tcp_sock(subflow), subflow, MPTCP_CF_FASTCLOSE); @@ -2885,6 +2871,24 @@ void mptcp_subflow_shutdown(struct sock *sk, struct sock *ssk, int how) release_sock(ssk); } +void mptcp_set_state(struct sock *sk, int state) +{ + int oldstate = sk->sk_state; + + switch (state) { + case TCP_ESTABLISHED: + if (oldstate != TCP_ESTABLISHED) + MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_CURRESTAB); + break; + + default: + if (oldstate == TCP_ESTABLISHED) + MPTCP_DEC_STATS(sock_net(sk), MPTCP_MIB_CURRESTAB); + } + + inet_sk_state_store(sk, state); +} + static const unsigned char new_state[16] = { /* current state: new state: action: */ [0 /* (Invalid) */] = TCP_CLOSE, @@ -2907,7 +2911,7 @@ static int mptcp_close_state(struct sock *sk) int next = (int)new_state[sk->sk_state]; int ns = next & TCP_STATE_MASK; - inet_sk_state_store(sk, ns); + mptcp_set_state(sk, ns); return next & TCP_ACTION_FIN; } @@ -3018,7 +3022,7 @@ bool __mptcp_close(struct sock *sk, long timeout) if ((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)) { mptcp_check_listen_stop(sk); - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); goto cleanup; } @@ -3061,7 +3065,7 @@ cleanup: * state, let's not keep resources busy for no reasons */ if (subflows_alive == 0) - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); sock_hold(sk); pr_debug("msk=%p state=%d", sk, sk->sk_state); @@ -3127,7 +3131,7 @@ static int mptcp_disconnect(struct sock *sk, int flags) return -EBUSY; mptcp_check_listen_stop(sk); - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); mptcp_stop_rtx_timer(sk); mptcp_stop_tout_timer(sk); @@ -3215,7 +3219,7 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk, /* this can't race with mptcp_close(), as the msk is * not yet exposted to user-space */ - inet_sk_state_store(nsk, TCP_ESTABLISHED); + mptcp_set_state(nsk, TCP_ESTABLISHED); /* The msk maintain a ref to each subflow in the connections list */ WRITE_ONCE(msk->first, ssk); @@ -3258,44 +3262,6 @@ void mptcp_rcv_space_init(struct mptcp_sock *msk, const struct sock *ssk) WRITE_ONCE(msk->wnd_end, msk->snd_nxt + tcp_sk(ssk)->snd_wnd); } -static struct sock *mptcp_accept(struct sock *ssk, int flags, int *err, - bool kern) -{ - struct sock *newsk; - - pr_debug("ssk=%p, listener=%p", ssk, mptcp_subflow_ctx(ssk)); - newsk = inet_csk_accept(ssk, flags, err, kern); - if (!newsk) - return NULL; - - pr_debug("newsk=%p, subflow is mptcp=%d", newsk, sk_is_mptcp(newsk)); - if (sk_is_mptcp(newsk)) { - struct mptcp_subflow_context *subflow; - struct sock *new_mptcp_sock; - - subflow = mptcp_subflow_ctx(newsk); - new_mptcp_sock = subflow->conn; - - /* is_mptcp should be false if subflow->conn is missing, see - * subflow_syn_recv_sock() - */ - if (WARN_ON_ONCE(!new_mptcp_sock)) { - tcp_sk(newsk)->is_mptcp = 0; - goto out; - } - - newsk = new_mptcp_sock; - MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_MPCAPABLEPASSIVEACK); - } else { - MPTCP_INC_STATS(sock_net(ssk), - MPTCP_MIB_MPCAPABLEPASSIVEFALLBACK); - } - -out: - newsk->sk_kern_sock = kern; - return newsk; -} - void mptcp_destroy_common(struct mptcp_sock *msk, unsigned int flags) { struct mptcp_subflow_context *subflow, *tmp; @@ -3674,7 +3640,7 @@ static int mptcp_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) if (IS_ERR(ssk)) return PTR_ERR(ssk); - inet_sk_state_store(sk, TCP_SYN_SENT); + mptcp_set_state(sk, TCP_SYN_SENT); subflow = mptcp_subflow_ctx(ssk); #ifdef CONFIG_TCP_MD5SIG /* no MPTCP if MD5SIG is enabled on this socket or we may run out of @@ -3724,7 +3690,7 @@ out: if (unlikely(err)) { /* avoid leaving a dangling token in an unconnected socket */ mptcp_token_destroy(msk); - inet_sk_state_store(sk, TCP_CLOSE); + mptcp_set_state(sk, TCP_CLOSE); return err; } @@ -3739,7 +3705,6 @@ static struct proto mptcp_prot = { .connect = mptcp_connect, .disconnect = mptcp_disconnect, .close = mptcp_close, - .accept = mptcp_accept, .setsockopt = mptcp_setsockopt, .getsockopt = mptcp_getsockopt, .shutdown = mptcp_shutdown, @@ -3814,13 +3779,13 @@ static int mptcp_listen(struct socket *sock, int backlog) goto unlock; } - inet_sk_state_store(sk, TCP_LISTEN); + mptcp_set_state(sk, TCP_LISTEN); sock_set_flag(sk, SOCK_RCU_FREE); lock_sock(ssk); err = __inet_listen_sk(ssk, backlog); release_sock(ssk); - inet_sk_state_store(sk, inet_sk_state_load(ssk)); + mptcp_set_state(sk, inet_sk_state_load(ssk)); if (!err) { sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1); @@ -3849,18 +3814,36 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock, if (!ssk) return -EINVAL; - newsk = mptcp_accept(ssk, flags, &err, kern); + pr_debug("ssk=%p, listener=%p", ssk, mptcp_subflow_ctx(ssk)); + newsk = inet_csk_accept(ssk, flags, &err, kern); if (!newsk) return err; - lock_sock(newsk); - - __inet_accept(sock, newsock, newsk); - if (!mptcp_is_tcpsk(newsock->sk)) { - struct mptcp_sock *msk = mptcp_sk(newsk); + pr_debug("newsk=%p, subflow is mptcp=%d", newsk, sk_is_mptcp(newsk)); + if (sk_is_mptcp(newsk)) { struct mptcp_subflow_context *subflow; + struct sock *new_mptcp_sock; + + subflow = mptcp_subflow_ctx(newsk); + new_mptcp_sock = subflow->conn; + + /* is_mptcp should be false if subflow->conn is missing, see + * subflow_syn_recv_sock() + */ + if (WARN_ON_ONCE(!new_mptcp_sock)) { + tcp_sk(newsk)->is_mptcp = 0; + goto tcpfallback; + } + + newsk = new_mptcp_sock; + MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_MPCAPABLEPASSIVEACK); + + newsk->sk_kern_sock = kern; + lock_sock(newsk); + __inet_accept(sock, newsock, newsk); set_bit(SOCK_CUSTOM_SOCKOPT, &newsock->flags); + msk = mptcp_sk(newsk); msk->in_accept_queue = 0; /* set ssk->sk_socket of accept()ed flows to mptcp socket. @@ -3880,8 +3863,23 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock, __mptcp_close_ssk(newsk, msk->first, mptcp_subflow_ctx(msk->first), 0); if (unlikely(list_is_singular(&msk->conn_list))) - inet_sk_state_store(newsk, TCP_CLOSE); + mptcp_set_state(newsk, TCP_CLOSE); } + } else { + MPTCP_INC_STATS(sock_net(ssk), + MPTCP_MIB_MPCAPABLEPASSIVEFALLBACK); +tcpfallback: + newsk->sk_kern_sock = kern; + lock_sock(newsk); + __inet_accept(sock, newsock, newsk); + /* we are being invoked after accepting a non-mp-capable + * flow: sk is a tcp_sk, not an mptcp one. + * + * Hand the socket over to tcp so all further socket ops + * bypass mptcp. + */ + WRITE_ONCE(newsock->sk->sk_socket->ops, + mptcp_fallback_tcp_ops(newsock->sk)); } release_sock(newsk); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 1240268f9e9e..3517f2d24a22 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -641,6 +641,7 @@ bool __mptcp_close(struct sock *sk, long timeout); void mptcp_cancel_work(struct sock *sk); void __mptcp_unaccepted_force_close(struct sock *sk); void mptcp_set_owner_r(struct sk_buff *skb, struct sock *sk); +void mptcp_set_state(struct sock *sk, int state); bool mptcp_addresses_equal(const struct mptcp_addr_info *a, const struct mptcp_addr_info *b, bool use_port); diff --git a/net/mptcp/sockopt.c b/net/mptcp/sockopt.c index cabe856b2a45..c40f1428e602 100644 --- a/net/mptcp/sockopt.c +++ b/net/mptcp/sockopt.c @@ -440,6 +440,8 @@ static bool mptcp_supported_sockopt(int level, int optname) /* should work fine */ case IP_FREEBIND: case IP_TRANSPARENT: + case IP_BIND_ADDRESS_NO_PORT: + case IP_LOCAL_PORT_RANGE: /* the following are control cmsg related */ case IP_PKTINFO: @@ -455,7 +457,6 @@ static bool mptcp_supported_sockopt(int level, int optname) /* common stuff that need some love */ case IP_TOS: case IP_TTL: - case IP_BIND_ADDRESS_NO_PORT: case IP_MTU_DISCOVER: case IP_RECVERR: @@ -683,8 +684,8 @@ static int mptcp_setsockopt_sol_tcp_nodelay(struct mptcp_sock *msk, sockptr_t op return 0; } -static int mptcp_setsockopt_sol_ip_set_transparent(struct mptcp_sock *msk, int optname, - sockptr_t optval, unsigned int optlen) +static int mptcp_setsockopt_sol_ip_set(struct mptcp_sock *msk, int optname, + sockptr_t optval, unsigned int optlen) { struct sock *sk = (struct sock *)msk; struct sock *ssk; @@ -710,6 +711,14 @@ static int mptcp_setsockopt_sol_ip_set_transparent(struct mptcp_sock *msk, int o inet_assign_bit(TRANSPARENT, ssk, inet_test_bit(TRANSPARENT, sk)); break; + case IP_BIND_ADDRESS_NO_PORT: + inet_assign_bit(BIND_ADDRESS_NO_PORT, ssk, + inet_test_bit(BIND_ADDRESS_NO_PORT, sk)); + break; + case IP_LOCAL_PORT_RANGE: + WRITE_ONCE(inet_sk(ssk)->local_port_range, + READ_ONCE(inet_sk(sk)->local_port_range)); + break; default: release_sock(sk); WARN_ON_ONCE(1); @@ -755,7 +764,9 @@ static int mptcp_setsockopt_v4(struct mptcp_sock *msk, int optname, switch (optname) { case IP_FREEBIND: case IP_TRANSPARENT: - return mptcp_setsockopt_sol_ip_set_transparent(msk, optname, optval, optlen); + case IP_BIND_ADDRESS_NO_PORT: + case IP_LOCAL_PORT_RANGE: + return mptcp_setsockopt_sol_ip_set(msk, optname, optval, optlen); case IP_TOS: return mptcp_setsockopt_v4_set_tos(msk, optname, optval, optlen); } @@ -1350,6 +1361,12 @@ static int mptcp_getsockopt_v4(struct mptcp_sock *msk, int optname, switch (optname) { case IP_TOS: return mptcp_put_int_option(msk, optval, optlen, READ_ONCE(inet_sk(sk)->tos)); + case IP_BIND_ADDRESS_NO_PORT: + return mptcp_put_int_option(msk, optval, optlen, + inet_test_bit(BIND_ADDRESS_NO_PORT, sk)); + case IP_LOCAL_PORT_RANGE: + return mptcp_put_int_option(msk, optval, optlen, + READ_ONCE(inet_sk(sk)->local_port_range)); } return -EOPNOTSUPP; @@ -1450,6 +1467,8 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk) inet_assign_bit(TRANSPARENT, ssk, inet_test_bit(TRANSPARENT, sk)); inet_assign_bit(FREEBIND, ssk, inet_test_bit(FREEBIND, sk)); + inet_assign_bit(BIND_ADDRESS_NO_PORT, ssk, inet_test_bit(BIND_ADDRESS_NO_PORT, sk)); + WRITE_ONCE(inet_sk(ssk)->local_port_range, READ_ONCE(inet_sk(sk)->local_port_range)); } void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk) diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 6d7684c35e93..1ef28642afc4 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -425,7 +425,7 @@ void __mptcp_sync_state(struct sock *sk, int state) __mptcp_propagate_sndbuf(sk, msk->first); if (sk->sk_state == TCP_SYN_SENT) { - inet_sk_state_store(sk, state); + mptcp_set_state(sk, state); sk->sk_state_change(sk); } } diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index fb0ae15e96df..0c22a02c2035 100644 --- a/net/netfilter/nf_conntrack_netlink.c +++ b/net/netfilter/nf_conntrack_netlink.c @@ -992,13 +992,13 @@ ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family) if (err) goto err_filter; - if (!cda[CTA_FILTER]) - return filter; - err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone); if (err < 0) goto err_filter; + if (!cda[CTA_FILTER]) + return filter; + err = ctnetlink_parse_filter(cda[CTA_FILTER], filter); if (err < 0) goto err_filter; @@ -1043,7 +1043,7 @@ err_filter: static bool ctnetlink_needs_filter(u8 family, const struct nlattr * const *cda) { - return family || cda[CTA_MARK] || cda[CTA_FILTER] || cda[CTA_STATUS]; + return family || cda[CTA_MARK] || cda[CTA_FILTER] || cda[CTA_STATUS] || cda[CTA_ZONE]; } static int ctnetlink_start(struct netlink_callback *cb) @@ -1148,6 +1148,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data) if (filter->family && nf_ct_l3num(ct) != filter->family) goto ignore_entry; + if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID && + !nf_ct_zone_equal_any(ct, &filter->zone)) + goto ignore_entry; + if (filter->orig_flags) { tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL); if (!ctnetlink_filter_match_tuple(&filter->orig, tuple, diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index c5c17c6e80ed..5531b13d92b6 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -2261,7 +2261,16 @@ static int nft_chain_parse_hook(struct net *net, return -EOPNOTSUPP; } - type = basechain->type; + if (nla[NFTA_CHAIN_TYPE]) { + type = __nf_tables_chain_type_lookup(nla[NFTA_CHAIN_TYPE], + family); + if (!type) { + NL_SET_BAD_ATTR(extack, nla[NFTA_CHAIN_TYPE]); + return -ENOENT; + } + } else { + type = basechain->type; + } } if (!try_module_get(type->owner)) { @@ -5817,10 +5826,6 @@ static int nf_tables_dump_set(struct sk_buff *skb, struct netlink_callback *cb) nla_nest_end(skb, nest); nlmsg_end(skb, nlh); - if (dump_ctx->reset && args.iter.count > args.iter.skip) - audit_log_nft_set_reset(table, cb->seq, - args.iter.count - args.iter.skip); - rcu_read_unlock(); if (args.iter.err && args.iter.err != -EMSGSIZE) @@ -5836,6 +5841,26 @@ nla_put_failure: return -ENOSPC; } +static int nf_tables_dumpreset_set(struct sk_buff *skb, + struct netlink_callback *cb) +{ + struct nftables_pernet *nft_net = nft_pernet(sock_net(skb->sk)); + struct nft_set_dump_ctx *dump_ctx = cb->data; + int ret, skip = cb->args[0]; + + mutex_lock(&nft_net->commit_mutex); + + ret = nf_tables_dump_set(skb, cb); + + if (cb->args[0] > skip) + audit_log_nft_set_reset(dump_ctx->ctx.table, cb->seq, + cb->args[0] - skip); + + mutex_unlock(&nft_net->commit_mutex); + + return ret; +} + static int nf_tables_dump_set_start(struct netlink_callback *cb) { struct nft_set_dump_ctx *dump_ctx = cb->data; @@ -5910,7 +5935,7 @@ static int nft_setelem_parse_flags(const struct nft_set *set, return 0; } -static int nft_setelem_parse_key(struct nft_ctx *ctx, struct nft_set *set, +static int nft_setelem_parse_key(struct nft_ctx *ctx, const struct nft_set *set, struct nft_data *key, struct nlattr *attr) { struct nft_data_desc desc = { @@ -5963,7 +5988,7 @@ static void *nft_setelem_catchall_get(const struct net *net, return priv; } -static int nft_setelem_get(struct nft_ctx *ctx, struct nft_set *set, +static int nft_setelem_get(struct nft_ctx *ctx, const struct nft_set *set, struct nft_set_elem *elem, u32 flags) { void *priv; @@ -5982,7 +6007,7 @@ static int nft_setelem_get(struct nft_ctx *ctx, struct nft_set *set, return 0; } -static int nft_get_set_elem(struct nft_ctx *ctx, struct nft_set *set, +static int nft_get_set_elem(struct nft_ctx *ctx, const struct nft_set *set, const struct nlattr *attr, bool reset) { struct nlattr *nla[NFTA_SET_ELEM_MAX + 1]; @@ -6039,21 +6064,18 @@ err_fill_setelem: return err; } -/* called with rcu_read_lock held */ -static int nf_tables_getsetelem(struct sk_buff *skb, - const struct nfnl_info *info, - const struct nlattr * const nla[]) +static int nft_set_dump_ctx_init(struct nft_set_dump_ctx *dump_ctx, + const struct sk_buff *skb, + const struct nfnl_info *info, + const struct nlattr * const nla[], + bool reset) { struct netlink_ext_ack *extack = info->extack; u8 genmask = nft_genmask_cur(info->net); u8 family = info->nfmsg->nfgen_family; - int rem, err = 0, nelems = 0; struct net *net = info->net; struct nft_table *table; struct nft_set *set; - struct nlattr *attr; - struct nft_ctx ctx; - bool reset = false; table = nft_table_lookup(net, nla[NFTA_SET_ELEM_LIST_TABLE], family, genmask, 0); @@ -6068,10 +6090,22 @@ static int nf_tables_getsetelem(struct sk_buff *skb, return PTR_ERR(set); } - nft_ctx_init(&ctx, net, skb, info->nlh, family, table, NULL, nla); + nft_ctx_init(&dump_ctx->ctx, net, skb, + info->nlh, family, table, NULL, nla); + dump_ctx->set = set; + dump_ctx->reset = reset; + return 0; +} - if (NFNL_MSG_TYPE(info->nlh->nlmsg_type) == NFT_MSG_GETSETELEM_RESET) - reset = true; +/* called with rcu_read_lock held */ +static int nf_tables_getsetelem(struct sk_buff *skb, + const struct nfnl_info *info, + const struct nlattr * const nla[]) +{ + struct netlink_ext_ack *extack = info->extack; + struct nft_set_dump_ctx dump_ctx; + struct nlattr *attr; + int rem, err = 0; if (info->nlh->nlmsg_flags & NLM_F_DUMP) { struct netlink_dump_control c = { @@ -6080,12 +6114,55 @@ static int nf_tables_getsetelem(struct sk_buff *skb, .done = nf_tables_dump_set_done, .module = THIS_MODULE, }; - struct nft_set_dump_ctx dump_ctx = { - .set = set, - .ctx = ctx, - .reset = reset, + + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, false); + if (err) + return err; + + c.data = &dump_ctx; + return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); + } + + if (!nla[NFTA_SET_ELEM_LIST_ELEMENTS]) + return -EINVAL; + + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, false); + if (err) + return err; + + nla_for_each_nested(attr, nla[NFTA_SET_ELEM_LIST_ELEMENTS], rem) { + err = nft_get_set_elem(&dump_ctx.ctx, dump_ctx.set, attr, false); + if (err < 0) { + NL_SET_BAD_ATTR(extack, attr); + break; + } + } + + return err; +} + +static int nf_tables_getsetelem_reset(struct sk_buff *skb, + const struct nfnl_info *info, + const struct nlattr * const nla[]) +{ + struct nftables_pernet *nft_net = nft_pernet(info->net); + struct netlink_ext_ack *extack = info->extack; + struct nft_set_dump_ctx dump_ctx; + int rem, err = 0, nelems = 0; + struct nlattr *attr; + + if (info->nlh->nlmsg_flags & NLM_F_DUMP) { + struct netlink_dump_control c = { + .start = nf_tables_dump_set_start, + .dump = nf_tables_dumpreset_set, + .done = nf_tables_dump_set_done, + .module = THIS_MODULE, }; + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, true); + if (err) + return err; + c.data = &dump_ctx; return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); } @@ -6093,18 +6170,31 @@ static int nf_tables_getsetelem(struct sk_buff *skb, if (!nla[NFTA_SET_ELEM_LIST_ELEMENTS]) return -EINVAL; + if (!try_module_get(THIS_MODULE)) + return -EINVAL; + rcu_read_unlock(); + mutex_lock(&nft_net->commit_mutex); + rcu_read_lock(); + + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, true); + if (err) + goto out_unlock; + nla_for_each_nested(attr, nla[NFTA_SET_ELEM_LIST_ELEMENTS], rem) { - err = nft_get_set_elem(&ctx, set, attr, reset); + err = nft_get_set_elem(&dump_ctx.ctx, dump_ctx.set, attr, true); if (err < 0) { NL_SET_BAD_ATTR(extack, attr); break; } nelems++; } + audit_log_nft_set_reset(dump_ctx.ctx.table, nft_net->base_seq, nelems); - if (reset) - audit_log_nft_set_reset(table, nft_pernet(net)->base_seq, - nelems); +out_unlock: + rcu_read_unlock(); + mutex_unlock(&nft_net->commit_mutex); + rcu_read_lock(); + module_put(THIS_MODULE); return err; } @@ -9078,7 +9168,7 @@ static const struct nfnl_callback nf_tables_cb[NFT_MSG_MAX] = { .policy = nft_set_elem_list_policy, }, [NFT_MSG_GETSETELEM_RESET] = { - .call = nf_tables_getsetelem, + .call = nf_tables_getsetelem_reset, .type = NFNL_CB_RCU, .attr_count = NFTA_SET_ELEM_LIST_MAX, .policy = nft_set_elem_list_policy, @@ -10383,6 +10473,7 @@ static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action) nft_trans_destroy(trans); break; } + nft_trans_set(trans)->dead = 1; list_del_rcu(&nft_trans_set(trans)->list); break; case NFT_MSG_DELSET: diff --git a/net/netfilter/nft_set_pipapo.c b/net/netfilter/nft_set_pipapo.c index 7252fcdae349..efd523496be4 100644 --- a/net/netfilter/nft_set_pipapo.c +++ b/net/netfilter/nft_set_pipapo.c @@ -1597,7 +1597,7 @@ static void pipapo_gc(struct nft_set *set, struct nft_pipapo_match *m) if (nft_set_elem_expired(&e->ext)) { priv->dirty = true; - gc = nft_trans_gc_queue_sync(gc, GFP_ATOMIC); + gc = nft_trans_gc_queue_sync(gc, GFP_KERNEL); if (!gc) return; diff --git a/net/netlink/genetlink.c b/net/netlink/genetlink.c index c0d15470a10b..8c7af02f8454 100644 --- a/net/netlink/genetlink.c +++ b/net/netlink/genetlink.c @@ -1829,10 +1829,10 @@ static int genl_bind(struct net *net, int group) continue; grp = &family->mcgrps[i]; - if ((grp->flags & GENL_UNS_ADMIN_PERM) && + if ((grp->flags & GENL_MCAST_CAP_NET_ADMIN) && !ns_capable(net->user_ns, CAP_NET_ADMIN)) ret = -EPERM; - if (grp->cap_sys_admin && + if ((grp->flags & GENL_MCAST_CAP_SYS_ADMIN) && !ns_capable(net->user_ns, CAP_SYS_ADMIN)) ret = -EPERM; diff --git a/net/psample/psample.c b/net/psample/psample.c index c34e902855db..ddd211a151d0 100644 --- a/net/psample/psample.c +++ b/net/psample/psample.c @@ -32,7 +32,7 @@ enum psample_nl_multicast_groups { static const struct genl_multicast_group psample_nl_mcgrps[] = { [PSAMPLE_NL_MCGRP_CONFIG] = { .name = PSAMPLE_NL_MCGRP_CONFIG_NAME }, [PSAMPLE_NL_MCGRP_SAMPLE] = { .name = PSAMPLE_NL_MCGRP_SAMPLE_NAME, - .flags = GENL_UNS_ADMIN_PERM }, + .flags = GENL_MCAST_CAP_NET_ADMIN, }, }; static struct genl_family psample_nl_family __ro_after_init; diff --git a/net/sched/Makefile b/net/sched/Makefile index b5fd49641d91..82c3f78ca486 100644 --- a/net/sched/Makefile +++ b/net/sched/Makefile @@ -13,7 +13,6 @@ obj-$(CONFIG_NET_ACT_POLICE) += act_police.o obj-$(CONFIG_NET_ACT_GACT) += act_gact.o obj-$(CONFIG_NET_ACT_MIRRED) += act_mirred.o obj-$(CONFIG_NET_ACT_SAMPLE) += act_sample.o -obj-$(CONFIG_NET_ACT_IPT) += act_ipt.o obj-$(CONFIG_NET_ACT_NAT) += act_nat.o obj-$(CONFIG_NET_ACT_PEDIT) += act_pedit.o obj-$(CONFIG_NET_ACT_SIMP) += act_simple.o diff --git a/net/sched/act_ipt.c b/net/sched/act_ipt.c deleted file mode 100644 index 598d6e299152..000000000000 --- a/net/sched/act_ipt.c +++ /dev/null @@ -1,464 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* - * net/sched/act_ipt.c iptables target interface - * - *TODO: Add other tables. For now we only support the ipv4 table targets - * - * Copyright: Jamal Hadi Salim (2002-13) - */ - -#include <linux/types.h> -#include <linux/kernel.h> -#include <linux/string.h> -#include <linux/errno.h> -#include <linux/skbuff.h> -#include <linux/rtnetlink.h> -#include <linux/module.h> -#include <linux/init.h> -#include <linux/slab.h> -#include <net/netlink.h> -#include <net/pkt_sched.h> -#include <linux/tc_act/tc_ipt.h> -#include <net/tc_act/tc_ipt.h> -#include <net/tc_wrapper.h> -#include <net/ip.h> - -#include <linux/netfilter_ipv4/ip_tables.h> - - -static struct tc_action_ops act_ipt_ops; -static struct tc_action_ops act_xt_ops; - -static int ipt_init_target(struct net *net, struct xt_entry_target *t, - char *table, unsigned int hook) -{ - struct xt_tgchk_param par; - struct xt_target *target; - struct ipt_entry e = {}; - int ret = 0; - - target = xt_request_find_target(AF_INET, t->u.user.name, - t->u.user.revision); - if (IS_ERR(target)) - return PTR_ERR(target); - - t->u.kernel.target = target; - memset(&par, 0, sizeof(par)); - par.net = net; - par.table = table; - par.entryinfo = &e; - par.target = target; - par.targinfo = t->data; - par.hook_mask = 1 << hook; - par.family = NFPROTO_IPV4; - - ret = xt_check_target(&par, t->u.target_size - sizeof(*t), 0, false); - if (ret < 0) { - module_put(t->u.kernel.target->me); - return ret; - } - return 0; -} - -static void ipt_destroy_target(struct xt_entry_target *t, struct net *net) -{ - struct xt_tgdtor_param par = { - .target = t->u.kernel.target, - .targinfo = t->data, - .family = NFPROTO_IPV4, - .net = net, - }; - if (par.target->destroy != NULL) - par.target->destroy(&par); - module_put(par.target->me); -} - -static void tcf_ipt_release(struct tc_action *a) -{ - struct tcf_ipt *ipt = to_ipt(a); - - if (ipt->tcfi_t) { - ipt_destroy_target(ipt->tcfi_t, a->idrinfo->net); - kfree(ipt->tcfi_t); - } - kfree(ipt->tcfi_tname); -} - -static const struct nla_policy ipt_policy[TCA_IPT_MAX + 1] = { - [TCA_IPT_TABLE] = { .type = NLA_STRING, .len = IFNAMSIZ }, - [TCA_IPT_HOOK] = NLA_POLICY_RANGE(NLA_U32, NF_INET_PRE_ROUTING, - NF_INET_NUMHOOKS), - [TCA_IPT_INDEX] = { .type = NLA_U32 }, - [TCA_IPT_TARG] = { .len = sizeof(struct xt_entry_target) }, -}; - -static int __tcf_ipt_init(struct net *net, unsigned int id, struct nlattr *nla, - struct nlattr *est, struct tc_action **a, - const struct tc_action_ops *ops, - struct tcf_proto *tp, u32 flags) -{ - struct tc_action_net *tn = net_generic(net, id); - bool bind = flags & TCA_ACT_FLAGS_BIND; - struct nlattr *tb[TCA_IPT_MAX + 1]; - struct tcf_ipt *ipt; - struct xt_entry_target *td, *t; - char *tname; - bool exists = false; - int ret = 0, err; - u32 hook = 0; - u32 index = 0; - - if (nla == NULL) - return -EINVAL; - - err = nla_parse_nested_deprecated(tb, TCA_IPT_MAX, nla, ipt_policy, - NULL); - if (err < 0) - return err; - - if (tb[TCA_IPT_INDEX] != NULL) - index = nla_get_u32(tb[TCA_IPT_INDEX]); - - err = tcf_idr_check_alloc(tn, &index, a, bind); - if (err < 0) - return err; - exists = err; - if (exists && bind) - return 0; - - if (tb[TCA_IPT_HOOK] == NULL || tb[TCA_IPT_TARG] == NULL) { - if (exists) - tcf_idr_release(*a, bind); - else - tcf_idr_cleanup(tn, index); - return -EINVAL; - } - - td = (struct xt_entry_target *)nla_data(tb[TCA_IPT_TARG]); - if (nla_len(tb[TCA_IPT_TARG]) != td->u.target_size) { - if (exists) - tcf_idr_release(*a, bind); - else - tcf_idr_cleanup(tn, index); - return -EINVAL; - } - - if (!exists) { - ret = tcf_idr_create(tn, index, est, a, ops, bind, - false, flags); - if (ret) { - tcf_idr_cleanup(tn, index); - return ret; - } - ret = ACT_P_CREATED; - } else { - if (bind)/* dont override defaults */ - return 0; - - if (!(flags & TCA_ACT_FLAGS_REPLACE)) { - tcf_idr_release(*a, bind); - return -EEXIST; - } - } - - err = -EINVAL; - hook = nla_get_u32(tb[TCA_IPT_HOOK]); - switch (hook) { - case NF_INET_PRE_ROUTING: - break; - case NF_INET_POST_ROUTING: - break; - default: - goto err1; - } - - if (tb[TCA_IPT_TABLE]) { - /* mangle only for now */ - if (nla_strcmp(tb[TCA_IPT_TABLE], "mangle")) - goto err1; - } - - tname = kstrdup("mangle", GFP_KERNEL); - if (unlikely(!tname)) - goto err1; - - t = kmemdup(td, td->u.target_size, GFP_KERNEL); - if (unlikely(!t)) - goto err2; - - err = ipt_init_target(net, t, tname, hook); - if (err < 0) - goto err3; - - ipt = to_ipt(*a); - - spin_lock_bh(&ipt->tcf_lock); - if (ret != ACT_P_CREATED) { - ipt_destroy_target(ipt->tcfi_t, net); - kfree(ipt->tcfi_tname); - kfree(ipt->tcfi_t); - } - ipt->tcfi_tname = tname; - ipt->tcfi_t = t; - ipt->tcfi_hook = hook; - spin_unlock_bh(&ipt->tcf_lock); - return ret; - -err3: - kfree(t); -err2: - kfree(tname); -err1: - tcf_idr_release(*a, bind); - return err; -} - -static int tcf_ipt_init(struct net *net, struct nlattr *nla, - struct nlattr *est, struct tc_action **a, - struct tcf_proto *tp, - u32 flags, struct netlink_ext_ack *extack) -{ - return __tcf_ipt_init(net, act_ipt_ops.net_id, nla, est, - a, &act_ipt_ops, tp, flags); -} - -static int tcf_xt_init(struct net *net, struct nlattr *nla, - struct nlattr *est, struct tc_action **a, - struct tcf_proto *tp, - u32 flags, struct netlink_ext_ack *extack) -{ - return __tcf_ipt_init(net, act_xt_ops.net_id, nla, est, - a, &act_xt_ops, tp, flags); -} - -static bool tcf_ipt_act_check(struct sk_buff *skb) -{ - const struct iphdr *iph; - unsigned int nhoff, len; - - if (!pskb_may_pull(skb, sizeof(struct iphdr))) - return false; - - nhoff = skb_network_offset(skb); - iph = ip_hdr(skb); - if (iph->ihl < 5 || iph->version != 4) - return false; - - len = skb_ip_totlen(skb); - if (skb->len < nhoff + len || len < (iph->ihl * 4u)) - return false; - - return pskb_may_pull(skb, iph->ihl * 4u); -} - -TC_INDIRECT_SCOPE int tcf_ipt_act(struct sk_buff *skb, - const struct tc_action *a, - struct tcf_result *res) -{ - char saved_cb[sizeof_field(struct sk_buff, cb)]; - int ret = 0, result = 0; - struct tcf_ipt *ipt = to_ipt(a); - struct xt_action_param par; - struct nf_hook_state state = { - .net = dev_net(skb->dev), - .in = skb->dev, - .hook = ipt->tcfi_hook, - .pf = NFPROTO_IPV4, - }; - - if (skb_protocol(skb, false) != htons(ETH_P_IP)) - return TC_ACT_UNSPEC; - - if (skb_unclone(skb, GFP_ATOMIC)) - return TC_ACT_UNSPEC; - - if (!tcf_ipt_act_check(skb)) - return TC_ACT_UNSPEC; - - if (state.hook == NF_INET_POST_ROUTING) { - if (!skb_dst(skb)) - return TC_ACT_UNSPEC; - - state.out = skb->dev; - } - - memcpy(saved_cb, skb->cb, sizeof(saved_cb)); - - spin_lock(&ipt->tcf_lock); - - tcf_lastuse_update(&ipt->tcf_tm); - bstats_update(&ipt->tcf_bstats, skb); - - /* yes, we have to worry about both in and out dev - * worry later - danger - this API seems to have changed - * from earlier kernels - */ - par.state = &state; - par.target = ipt->tcfi_t->u.kernel.target; - par.targinfo = ipt->tcfi_t->data; - - memset(IPCB(skb), 0, sizeof(struct inet_skb_parm)); - - ret = par.target->target(skb, &par); - - switch (ret) { - case NF_ACCEPT: - result = TC_ACT_OK; - break; - case NF_DROP: - result = TC_ACT_SHOT; - ipt->tcf_qstats.drops++; - break; - case XT_CONTINUE: - result = TC_ACT_PIPE; - break; - default: - net_notice_ratelimited("tc filter: Bogus netfilter code %d assume ACCEPT\n", - ret); - result = TC_ACT_OK; - break; - } - spin_unlock(&ipt->tcf_lock); - - memcpy(skb->cb, saved_cb, sizeof(skb->cb)); - - return result; - -} - -static int tcf_ipt_dump(struct sk_buff *skb, struct tc_action *a, int bind, - int ref) -{ - unsigned char *b = skb_tail_pointer(skb); - struct tcf_ipt *ipt = to_ipt(a); - struct xt_entry_target *t; - struct tcf_t tm; - struct tc_cnt c; - - /* for simple targets kernel size == user size - * user name = target name - * for foolproof you need to not assume this - */ - - spin_lock_bh(&ipt->tcf_lock); - t = kmemdup(ipt->tcfi_t, ipt->tcfi_t->u.user.target_size, GFP_ATOMIC); - if (unlikely(!t)) - goto nla_put_failure; - - c.bindcnt = atomic_read(&ipt->tcf_bindcnt) - bind; - c.refcnt = refcount_read(&ipt->tcf_refcnt) - ref; - strcpy(t->u.user.name, ipt->tcfi_t->u.kernel.target->name); - - if (nla_put(skb, TCA_IPT_TARG, ipt->tcfi_t->u.user.target_size, t) || - nla_put_u32(skb, TCA_IPT_INDEX, ipt->tcf_index) || - nla_put_u32(skb, TCA_IPT_HOOK, ipt->tcfi_hook) || - nla_put(skb, TCA_IPT_CNT, sizeof(struct tc_cnt), &c) || - nla_put_string(skb, TCA_IPT_TABLE, ipt->tcfi_tname)) - goto nla_put_failure; - - tcf_tm_dump(&tm, &ipt->tcf_tm); - if (nla_put_64bit(skb, TCA_IPT_TM, sizeof(tm), &tm, TCA_IPT_PAD)) - goto nla_put_failure; - - spin_unlock_bh(&ipt->tcf_lock); - kfree(t); - return skb->len; - -nla_put_failure: - spin_unlock_bh(&ipt->tcf_lock); - nlmsg_trim(skb, b); - kfree(t); - return -1; -} - -static struct tc_action_ops act_ipt_ops = { - .kind = "ipt", - .id = TCA_ID_IPT, - .owner = THIS_MODULE, - .act = tcf_ipt_act, - .dump = tcf_ipt_dump, - .cleanup = tcf_ipt_release, - .init = tcf_ipt_init, - .size = sizeof(struct tcf_ipt), -}; - -static __net_init int ipt_init_net(struct net *net) -{ - struct tc_action_net *tn = net_generic(net, act_ipt_ops.net_id); - - return tc_action_net_init(net, tn, &act_ipt_ops); -} - -static void __net_exit ipt_exit_net(struct list_head *net_list) -{ - tc_action_net_exit(net_list, act_ipt_ops.net_id); -} - -static struct pernet_operations ipt_net_ops = { - .init = ipt_init_net, - .exit_batch = ipt_exit_net, - .id = &act_ipt_ops.net_id, - .size = sizeof(struct tc_action_net), -}; - -static struct tc_action_ops act_xt_ops = { - .kind = "xt", - .id = TCA_ID_XT, - .owner = THIS_MODULE, - .act = tcf_ipt_act, - .dump = tcf_ipt_dump, - .cleanup = tcf_ipt_release, - .init = tcf_xt_init, - .size = sizeof(struct tcf_ipt), -}; - -static __net_init int xt_init_net(struct net *net) -{ - struct tc_action_net *tn = net_generic(net, act_xt_ops.net_id); - - return tc_action_net_init(net, tn, &act_xt_ops); -} - -static void __net_exit xt_exit_net(struct list_head *net_list) -{ - tc_action_net_exit(net_list, act_xt_ops.net_id); -} - -static struct pernet_operations xt_net_ops = { - .init = xt_init_net, - .exit_batch = xt_exit_net, - .id = &act_xt_ops.net_id, - .size = sizeof(struct tc_action_net), -}; - -MODULE_AUTHOR("Jamal Hadi Salim(2002-13)"); -MODULE_DESCRIPTION("Iptables target actions"); -MODULE_LICENSE("GPL"); -MODULE_ALIAS("act_xt"); - -static int __init ipt_init_module(void) -{ - int ret1, ret2; - - ret1 = tcf_register_action(&act_xt_ops, &xt_net_ops); - if (ret1 < 0) - pr_err("Failed to load xt action\n"); - - ret2 = tcf_register_action(&act_ipt_ops, &ipt_net_ops); - if (ret2 < 0) - pr_err("Failed to load ipt action\n"); - - if (ret1 < 0 && ret2 < 0) { - return ret1; - } else - return 0; -} - -static void __exit ipt_cleanup_module(void) -{ - tcf_unregister_action(&act_ipt_ops, &ipt_net_ops); - tcf_unregister_action(&act_xt_ops, &xt_net_ops); -} - -module_init(ipt_init_module); -module_exit(ipt_cleanup_module); diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c index 0a711c184c29..d1f9794ca9b7 100644 --- a/net/sched/act_mirred.c +++ b/net/sched/act_mirred.c @@ -85,10 +85,21 @@ static void tcf_mirred_release(struct tc_action *a) static const struct nla_policy mirred_policy[TCA_MIRRED_MAX + 1] = { [TCA_MIRRED_PARMS] = { .len = sizeof(struct tc_mirred) }, + [TCA_MIRRED_BLOCKID] = NLA_POLICY_MIN(NLA_U32, 1), }; static struct tc_action_ops act_mirred_ops; +static void tcf_mirred_replace_dev(struct tcf_mirred *m, + struct net_device *ndev) +{ + struct net_device *odev; + + odev = rcu_replace_pointer(m->tcfm_dev, ndev, + lockdep_is_held(&m->tcf_lock)); + netdev_put(odev, &m->tcfm_dev_tracker); +} + static int tcf_mirred_init(struct net *net, struct nlattr *nla, struct nlattr *est, struct tc_action **a, struct tcf_proto *tp, @@ -126,6 +137,17 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, if (exists && bind) return 0; + if (tb[TCA_MIRRED_BLOCKID] && parm->ifindex) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot specify Block ID and dev simultaneously"); + if (exists) + tcf_idr_release(*a, bind); + else + tcf_idr_cleanup(tn, index); + + return -EINVAL; + } + switch (parm->eaction) { case TCA_EGRESS_MIRROR: case TCA_EGRESS_REDIR: @@ -142,9 +164,10 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, } if (!exists) { - if (!parm->ifindex) { + if (!parm->ifindex && !tb[TCA_MIRRED_BLOCKID]) { tcf_idr_cleanup(tn, index); - NL_SET_ERR_MSG_MOD(extack, "Specified device does not exist"); + NL_SET_ERR_MSG_MOD(extack, + "Must specify device or block"); return -EINVAL; } ret = tcf_idr_create_from_flags(tn, index, est, a, @@ -170,7 +193,7 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, spin_lock_bh(&m->tcf_lock); if (parm->ifindex) { - struct net_device *odev, *ndev; + struct net_device *ndev; ndev = dev_get_by_index(net, parm->ifindex); if (!ndev) { @@ -179,11 +202,14 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, goto put_chain; } mac_header_xmit = dev_is_mac_header_xmit(ndev); - odev = rcu_replace_pointer(m->tcfm_dev, ndev, - lockdep_is_held(&m->tcf_lock)); - netdev_put(odev, &m->tcfm_dev_tracker); + tcf_mirred_replace_dev(m, ndev); netdev_tracker_alloc(ndev, &m->tcfm_dev_tracker, GFP_ATOMIC); m->tcfm_mac_header_xmit = mac_header_xmit; + m->tcfm_blockid = 0; + } else if (tb[TCA_MIRRED_BLOCKID]) { + tcf_mirred_replace_dev(m, NULL); + m->tcfm_mac_header_xmit = false; + m->tcfm_blockid = nla_get_u32(tb[TCA_MIRRED_BLOCKID]); } goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch); m->tcfm_eaction = parm->eaction; @@ -225,48 +251,26 @@ static int tcf_mirred_forward(bool want_ingress, struct sk_buff *skb) return err; } -TC_INDIRECT_SCOPE int tcf_mirred_act(struct sk_buff *skb, - const struct tc_action *a, - struct tcf_result *res) +static int tcf_mirred_to_dev(struct sk_buff *skb, struct tcf_mirred *m, + struct net_device *dev, + const bool m_mac_header_xmit, int m_eaction, + int retval) { - struct tcf_mirred *m = to_mirred(a); - struct sk_buff *skb2 = skb; - bool m_mac_header_xmit; - struct net_device *dev; - unsigned int nest_level; - int retval, err = 0; - bool use_reinsert; + struct sk_buff *skb_to_send = skb; bool want_ingress; bool is_redirect; bool expects_nh; bool at_ingress; - int m_eaction; + bool dont_clone; int mac_len; bool at_nh; + int err; - nest_level = __this_cpu_inc_return(mirred_nest_level); - if (unlikely(nest_level > MIRRED_NEST_LIMIT)) { - net_warn_ratelimited("Packet exceeded mirred recursion limit on dev %s\n", - netdev_name(skb->dev)); - __this_cpu_dec(mirred_nest_level); - return TC_ACT_SHOT; - } - - tcf_lastuse_update(&m->tcf_tm); - tcf_action_update_bstats(&m->common, skb); - - m_mac_header_xmit = READ_ONCE(m->tcfm_mac_header_xmit); - m_eaction = READ_ONCE(m->tcfm_eaction); - retval = READ_ONCE(m->tcf_action); - dev = rcu_dereference_bh(m->tcfm_dev); - if (unlikely(!dev)) { - pr_notice_once("tc mirred: target device is gone\n"); - goto out; - } - + is_redirect = tcf_mirred_is_act_redirect(m_eaction); if (unlikely(!(dev->flags & IFF_UP)) || !netif_carrier_ok(dev)) { net_notice_ratelimited("tc mirred to Houston: device %s is down\n", dev->name); + err = -ENODEV; goto out; } @@ -274,61 +278,188 @@ TC_INDIRECT_SCOPE int tcf_mirred_act(struct sk_buff *skb, * since we can't easily detect the clsact caller, skip clone only for * ingress - that covers the TC S/W datapath. */ - is_redirect = tcf_mirred_is_act_redirect(m_eaction); at_ingress = skb_at_tc_ingress(skb); - use_reinsert = at_ingress && is_redirect && - tcf_mirred_can_reinsert(retval); - if (!use_reinsert) { - skb2 = skb_clone(skb, GFP_ATOMIC); - if (!skb2) + dont_clone = skb_at_tc_ingress(skb) && is_redirect && + tcf_mirred_can_reinsert(retval); + if (!dont_clone) { + skb_to_send = skb_clone(skb, GFP_ATOMIC); + if (!skb_to_send) { + err = -ENOMEM; goto out; + } } want_ingress = tcf_mirred_act_wants_ingress(m_eaction); /* All mirred/redirected skbs should clear previous ct info */ - nf_reset_ct(skb2); + nf_reset_ct(skb_to_send); if (want_ingress && !at_ingress) /* drop dst for egress -> ingress */ - skb_dst_drop(skb2); + skb_dst_drop(skb_to_send); expects_nh = want_ingress || !m_mac_header_xmit; at_nh = skb->data == skb_network_header(skb); if (at_nh != expects_nh) { - mac_len = skb_at_tc_ingress(skb) ? skb->mac_len : + mac_len = at_ingress ? skb->mac_len : skb_network_offset(skb); if (expects_nh) { /* target device/action expect data at nh */ - skb_pull_rcsum(skb2, mac_len); + skb_pull_rcsum(skb_to_send, mac_len); } else { /* target device/action expect data at mac */ - skb_push_rcsum(skb2, mac_len); + skb_push_rcsum(skb_to_send, mac_len); } } - skb2->skb_iif = skb->dev->ifindex; - skb2->dev = dev; + skb_to_send->skb_iif = skb->dev->ifindex; + skb_to_send->dev = dev; - /* mirror is always swallowed */ if (is_redirect) { - skb_set_redirected(skb2, skb2->tc_at_ingress); - - /* let's the caller reinsert the packet, if possible */ - if (use_reinsert) { - err = tcf_mirred_forward(want_ingress, skb); - if (err) - tcf_action_inc_overlimit_qstats(&m->common); - __this_cpu_dec(mirred_nest_level); - return TC_ACT_CONSUMED; - } + if (skb == skb_to_send) + retval = TC_ACT_CONSUMED; + + skb_set_redirected(skb_to_send, skb_to_send->tc_at_ingress); + + err = tcf_mirred_forward(want_ingress, skb_to_send); + } else { + err = tcf_mirred_forward(want_ingress, skb_to_send); } - err = tcf_mirred_forward(want_ingress, skb2); if (err) { out: tcf_action_inc_overlimit_qstats(&m->common); - if (tcf_mirred_is_act_redirect(m_eaction)) + if (is_redirect) retval = TC_ACT_SHOT; } + + return retval; +} + +static int tcf_blockcast_redir(struct sk_buff *skb, struct tcf_mirred *m, + struct tcf_block *block, int m_eaction, + const u32 exception_ifindex, int retval) +{ + struct net_device *dev_prev = NULL; + struct net_device *dev = NULL; + unsigned long index; + int mirred_eaction; + + mirred_eaction = tcf_mirred_act_wants_ingress(m_eaction) ? + TCA_INGRESS_MIRROR : TCA_EGRESS_MIRROR; + + xa_for_each(&block->ports, index, dev) { + if (index == exception_ifindex) + continue; + + if (!dev_prev) + goto assign_prev; + + tcf_mirred_to_dev(skb, m, dev_prev, + dev_is_mac_header_xmit(dev), + mirred_eaction, retval); +assign_prev: + dev_prev = dev; + } + + if (dev_prev) + return tcf_mirred_to_dev(skb, m, dev_prev, + dev_is_mac_header_xmit(dev_prev), + m_eaction, retval); + + return retval; +} + +static int tcf_blockcast_mirror(struct sk_buff *skb, struct tcf_mirred *m, + struct tcf_block *block, int m_eaction, + const u32 exception_ifindex, int retval) +{ + struct net_device *dev = NULL; + unsigned long index; + + xa_for_each(&block->ports, index, dev) { + if (index == exception_ifindex) + continue; + + tcf_mirred_to_dev(skb, m, dev, + dev_is_mac_header_xmit(dev), + m_eaction, retval); + } + + return retval; +} + +static int tcf_blockcast(struct sk_buff *skb, struct tcf_mirred *m, + const u32 blockid, struct tcf_result *res, + int retval) +{ + const u32 exception_ifindex = skb->dev->ifindex; + struct tcf_block *block; + bool is_redirect; + int m_eaction; + + m_eaction = READ_ONCE(m->tcfm_eaction); + is_redirect = tcf_mirred_is_act_redirect(m_eaction); + + /* we are already under rcu protection, so can call block lookup + * directly. + */ + block = tcf_block_lookup(dev_net(skb->dev), blockid); + if (!block || xa_empty(&block->ports)) { + tcf_action_inc_overlimit_qstats(&m->common); + return retval; + } + + if (is_redirect) + return tcf_blockcast_redir(skb, m, block, m_eaction, + exception_ifindex, retval); + + /* If it's not redirect, it is mirror */ + return tcf_blockcast_mirror(skb, m, block, m_eaction, exception_ifindex, + retval); +} + +TC_INDIRECT_SCOPE int tcf_mirred_act(struct sk_buff *skb, + const struct tc_action *a, + struct tcf_result *res) +{ + struct tcf_mirred *m = to_mirred(a); + int retval = READ_ONCE(m->tcf_action); + unsigned int nest_level; + bool m_mac_header_xmit; + struct net_device *dev; + int m_eaction; + u32 blockid; + + nest_level = __this_cpu_inc_return(mirred_nest_level); + if (unlikely(nest_level > MIRRED_NEST_LIMIT)) { + net_warn_ratelimited("Packet exceeded mirred recursion limit on dev %s\n", + netdev_name(skb->dev)); + retval = TC_ACT_SHOT; + goto dec_nest_level; + } + + tcf_lastuse_update(&m->tcf_tm); + tcf_action_update_bstats(&m->common, skb); + + blockid = READ_ONCE(m->tcfm_blockid); + if (blockid) { + retval = tcf_blockcast(skb, m, blockid, res, retval); + goto dec_nest_level; + } + + dev = rcu_dereference_bh(m->tcfm_dev); + if (unlikely(!dev)) { + pr_notice_once("tc mirred: target device is gone\n"); + tcf_action_inc_overlimit_qstats(&m->common); + goto dec_nest_level; + } + + m_mac_header_xmit = READ_ONCE(m->tcfm_mac_header_xmit); + m_eaction = READ_ONCE(m->tcfm_eaction); + + retval = tcf_mirred_to_dev(skb, m, dev, m_mac_header_xmit, m_eaction, + retval); + +dec_nest_level: __this_cpu_dec(mirred_nest_level); return retval; @@ -356,6 +487,7 @@ static int tcf_mirred_dump(struct sk_buff *skb, struct tc_action *a, int bind, }; struct net_device *dev; struct tcf_t t; + u32 blockid; spin_lock_bh(&m->tcf_lock); opt.action = m->tcf_action; @@ -367,6 +499,10 @@ static int tcf_mirred_dump(struct sk_buff *skb, struct tc_action *a, int bind, if (nla_put(skb, TCA_MIRRED_PARMS, sizeof(opt), &opt)) goto nla_put_failure; + blockid = m->tcfm_blockid; + if (blockid && nla_put_u32(skb, TCA_MIRRED_BLOCKID, blockid)) + goto nla_put_failure; + tcf_tm_dump(&t, &m->tcf_tm); if (nla_put_64bit(skb, TCA_MIRRED_TM, sizeof(t), &t, TCA_MIRRED_PAD)) goto nla_put_failure; @@ -397,6 +533,8 @@ static int mirred_device_event(struct notifier_block *unused, * net_device are already rcu protected. */ RCU_INIT_POINTER(m->tcfm_dev, NULL); + } else if (m->tcfm_blockid) { + m->tcfm_blockid = 0; } spin_unlock_bh(&m->tcf_lock); } diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index 8978cf5531d0..3c50b4037755 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -531,6 +531,7 @@ static void tcf_block_destroy(struct tcf_block *block) { mutex_destroy(&block->lock); mutex_destroy(&block->proto_destroy_lock); + xa_destroy(&block->ports); kfree_rcu(block, rcu); } @@ -1002,6 +1003,7 @@ static struct tcf_block *tcf_block_create(struct net *net, struct Qdisc *q, refcount_set(&block->refcnt, 1); block->net = net; block->index = block_index; + xa_init(&block->ports); /* Don't store q pointer for blocks which are shared */ if (!tcf_block_shared(block)) @@ -1009,12 +1011,13 @@ static struct tcf_block *tcf_block_create(struct net *net, struct Qdisc *q, return block; } -static struct tcf_block *tcf_block_lookup(struct net *net, u32 block_index) +struct tcf_block *tcf_block_lookup(struct net *net, u32 block_index) { struct tcf_net *tn = net_generic(net, tcf_net_id); return idr_find(&tn->idr, block_index); } +EXPORT_SYMBOL(tcf_block_lookup); static struct tcf_block *tcf_block_refcnt_get(struct net *net, u32 block_index) { diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index e9eaf637220e..299086bb6205 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -1180,6 +1180,43 @@ skip: return 0; } +static int qdisc_block_add_dev(struct Qdisc *sch, struct net_device *dev, + struct netlink_ext_ack *extack) +{ + const struct Qdisc_class_ops *cl_ops = sch->ops->cl_ops; + struct tcf_block *block; + int err; + + block = cl_ops->tcf_block(sch, TC_H_MIN_INGRESS, NULL); + if (block) { + err = xa_insert(&block->ports, dev->ifindex, dev, GFP_KERNEL); + if (err) { + NL_SET_ERR_MSG(extack, + "ingress block dev insert failed"); + return err; + } + } + + block = cl_ops->tcf_block(sch, TC_H_MIN_EGRESS, NULL); + if (block) { + err = xa_insert(&block->ports, dev->ifindex, dev, GFP_KERNEL); + if (err) { + NL_SET_ERR_MSG(extack, + "Egress block dev insert failed"); + goto err_out; + } + } + + return 0; + +err_out: + block = cl_ops->tcf_block(sch, TC_H_MIN_INGRESS, NULL); + if (block) + xa_erase(&block->ports, dev->ifindex); + + return err; +} + static int qdisc_block_indexes_set(struct Qdisc *sch, struct nlattr **tca, struct netlink_ext_ack *extack) { @@ -1350,6 +1387,10 @@ static struct Qdisc *qdisc_create(struct net_device *dev, qdisc_hash_add(sch, false); trace_qdisc_create(ops, dev, parent); + err = qdisc_block_add_dev(sch, dev, extack); + if (err) + goto err_out4; + return sch; err_out4: diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 8dd0e5925342..e33568df97a5 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -1051,6 +1051,9 @@ static void qdisc_free_cb(struct rcu_head *head) static void __qdisc_destroy(struct Qdisc *qdisc) { const struct Qdisc_ops *ops = qdisc->ops; + struct net_device *dev = qdisc_dev(qdisc); + const struct Qdisc_class_ops *cops; + struct tcf_block *block; #ifdef CONFIG_NET_SCHED qdisc_hash_del(qdisc); @@ -1061,11 +1064,24 @@ static void __qdisc_destroy(struct Qdisc *qdisc) qdisc_reset(qdisc); + cops = ops->cl_ops; + if (ops->ingress_block_get) { + block = cops->tcf_block(qdisc, TC_H_MIN_INGRESS, NULL); + if (block) + xa_erase(&block->ports, dev->ifindex); + } + + if (ops->egress_block_get) { + block = cops->tcf_block(qdisc, TC_H_MIN_EGRESS, NULL); + if (block) + xa_erase(&block->ports, dev->ifindex); + } + if (ops->destroy) ops->destroy(qdisc); module_put(ops->owner); - netdev_put(qdisc_dev(qdisc), &qdisc->dev_tracker); + netdev_put(dev, &qdisc->dev_tracker); trace_qdisc_destroy(qdisc); diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 7fc2f3c6d248..a2cb30af46cb 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -677,8 +677,6 @@ static bool smc_isascii(char *hostname) static void smc_conn_save_peer_info_fce(struct smc_sock *smc, struct smc_clc_msg_accept_confirm *clc) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)clc; struct smc_clc_first_contact_ext *fce; int clc_v2_len; @@ -687,17 +685,15 @@ static void smc_conn_save_peer_info_fce(struct smc_sock *smc, return; if (smc->conn.lgr->is_smcd) { - memcpy(smc->conn.lgr->negotiated_eid, clc_v2->d1.eid, + memcpy(smc->conn.lgr->negotiated_eid, clc->d1.eid, SMC_MAX_EID_LEN); - clc_v2_len = offsetofend(struct smc_clc_msg_accept_confirm_v2, - d1); + clc_v2_len = offsetofend(struct smc_clc_msg_accept_confirm, d1); } else { - memcpy(smc->conn.lgr->negotiated_eid, clc_v2->r1.eid, + memcpy(smc->conn.lgr->negotiated_eid, clc->r1.eid, SMC_MAX_EID_LEN); - clc_v2_len = offsetofend(struct smc_clc_msg_accept_confirm_v2, - r1); + clc_v2_len = offsetofend(struct smc_clc_msg_accept_confirm, r1); } - fce = (struct smc_clc_first_contact_ext *)(((u8 *)clc_v2) + clc_v2_len); + fce = (struct smc_clc_first_contact_ext *)(((u8 *)clc) + clc_v2_len); smc->conn.lgr->peer_os = fce->os_type; smc->conn.lgr->peer_smc_release = fce->release; if (smc_isascii(fce->hostname)) @@ -1048,7 +1044,8 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, { int rc = SMC_CLC_DECL_NOSMCDDEV; struct smcd_dev *smcd; - int i = 1; + int i = 1, entry = 1; + bool is_virtual; u16 chid; if (smcd_indicated(ini->smc_type_v1)) @@ -1060,14 +1057,23 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, chid = smc_ism_get_chid(smcd); if (!smc_find_ism_v2_is_unique_chid(chid, ini, i)) continue; + is_virtual = __smc_ism_is_virtual(chid); if (!smc_pnet_is_pnetid_set(smcd->pnetid) || smc_pnet_is_ndev_pnetid(sock_net(&smc->sk), smcd->pnetid)) { + if (is_virtual && entry == SMCD_CLC_MAX_V2_GID_ENTRIES) + /* It's the last GID-CHID entry left in CLC + * Proposal SMC-Dv2 extension, but a virtual + * ISM device will take two entries. So give + * up it and try the next potential ISM device. + */ + continue; ini->ism_dev[i] = smcd; ini->ism_chid[i] = chid; ini->is_smcd = true; rc = 0; i++; - if (i > SMC_MAX_ISM_DEVS) + entry = is_virtual ? entry + 2 : entry + 1; + if (entry > SMCD_CLC_MAX_V2_GID_ENTRIES) break; } } @@ -1149,13 +1155,13 @@ static int smc_connect_ism_vlan_cleanup(struct smc_sock *smc, } #define SMC_CLC_MAX_ACCEPT_LEN \ - (sizeof(struct smc_clc_msg_accept_confirm_v2) + \ + (sizeof(struct smc_clc_msg_accept_confirm) + \ sizeof(struct smc_clc_first_contact_ext_v2x) + \ sizeof(struct smc_clc_msg_trail)) /* CLC handshake during connect */ static int smc_connect_clc(struct smc_sock *smc, - struct smc_clc_msg_accept_confirm_v2 *aclc2, + struct smc_clc_msg_accept_confirm *aclc, struct smc_init_info *ini) { int rc = 0; @@ -1165,7 +1171,7 @@ static int smc_connect_clc(struct smc_sock *smc, if (rc) return rc; /* receive SMC Accept CLC message */ - return smc_clc_wait_msg(smc, aclc2, SMC_CLC_MAX_ACCEPT_LEN, + return smc_clc_wait_msg(smc, aclc, SMC_CLC_MAX_ACCEPT_LEN, SMC_CLC_ACCEPT, CLC_WAIT_TIME); } @@ -1201,10 +1207,8 @@ static int smc_connect_rdma_v2_prepare(struct smc_sock *smc, struct smc_clc_msg_accept_confirm *aclc, struct smc_init_info *ini) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)aclc; struct smc_clc_first_contact_ext *fce = - smc_get_clc_first_contact_ext(clc_v2, false); + smc_get_clc_first_contact_ext(aclc, false); struct net *net = sock_net(&smc->sk); int rc; @@ -1327,10 +1331,7 @@ static int smc_connect_rdma(struct smc_sock *smc, } if (aclc->hdr.version > SMC_V1) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)aclc; - - eid = clc_v2->r1.eid; + eid = aclc->r1.eid; if (ini->first_contact_local) smc_fill_gid_list(link->lgr, &ini->smcrv2.gidlist, link->smcibdev, link->gid); @@ -1371,7 +1372,7 @@ connect_abort: * Determine from the CHID of the received CLC ACCEPT the ISM device chosen. */ static int -smc_v2_determine_accepted_chid(struct smc_clc_msg_accept_confirm_v2 *aclc, +smc_v2_determine_accepted_chid(struct smc_clc_msg_accept_confirm *aclc, struct smc_init_info *ini) { int i; @@ -1398,12 +1399,9 @@ static int smc_connect_ism(struct smc_sock *smc, ini->first_contact_peer = aclc->hdr.typev2 & SMC_FIRST_CONTACT_MASK; if (aclc->hdr.version == SMC_V2) { - struct smc_clc_msg_accept_confirm_v2 *aclc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)aclc; - if (ini->first_contact_peer) { struct smc_clc_first_contact_ext *fce = - smc_get_clc_first_contact_ext(aclc_v2, true); + smc_get_clc_first_contact_ext(aclc, true); ini->release_nr = fce->release; rc = smc_clc_clnt_v2x_features_validate(fce, ini); @@ -1411,11 +1409,16 @@ static int smc_connect_ism(struct smc_sock *smc, return rc; } - rc = smc_v2_determine_accepted_chid(aclc_v2, ini); + rc = smc_v2_determine_accepted_chid(aclc, ini); if (rc) return rc; + + if (__smc_ism_is_virtual(ini->ism_chid[ini->ism_selected])) + ini->ism_peer_gid[ini->ism_selected].gid_ext = + ntohll(aclc->d1.gid_ext); + /* for non-virtual ISM devices, peer gid_ext remains 0. */ } - ini->ism_peer_gid[ini->ism_selected] = ntohll(aclc->d0.gid); + ini->ism_peer_gid[ini->ism_selected].gid = ntohll(aclc->d0.gid); /* there is only one lgr role for SMC-D; use server lock */ mutex_lock(&smc_server_lgr_pending); @@ -1437,12 +1440,8 @@ static int smc_connect_ism(struct smc_sock *smc, smc_rx_init(smc); smc_tx_init(smc); - if (aclc->hdr.version > SMC_V1) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)aclc; - - eid = clc_v2->d1.eid; - } + if (aclc->hdr.version > SMC_V1) + eid = aclc->d1.eid; rc = smc_clc_send_confirm(smc, ini->first_contact_local, aclc->hdr.version, eid, ini); @@ -1493,7 +1492,6 @@ static int smc_connect_check_aclc(struct smc_init_info *ini, static int __smc_connect(struct smc_sock *smc) { u8 version = smc_ism_is_v2_capable() ? SMC_V2 : SMC_V1; - struct smc_clc_msg_accept_confirm_v2 *aclc2; struct smc_clc_msg_accept_confirm *aclc; struct smc_init_info *ini = NULL; u8 *buf = NULL; @@ -1541,11 +1539,10 @@ static int __smc_connect(struct smc_sock *smc) rc = SMC_CLC_DECL_MEM; goto fallback; } - aclc2 = (struct smc_clc_msg_accept_confirm_v2 *)buf; - aclc = (struct smc_clc_msg_accept_confirm *)aclc2; + aclc = (struct smc_clc_msg_accept_confirm *)buf; /* perform CLC handshake */ - rc = smc_connect_clc(smc, aclc2, ini); + rc = smc_connect_clc(smc, aclc, ini); if (rc) { /* -EAGAIN on timeout, see tcp_recvmsg() */ if (rc == -EAGAIN) { @@ -2106,7 +2103,8 @@ static bool smc_is_already_selected(struct smcd_dev *smcd, /* check for ISM devices matching proposed ISM devices */ static void smc_check_ism_v2_match(struct smc_init_info *ini, - u16 proposed_chid, u64 proposed_gid, + u16 proposed_chid, + struct smcd_gid *proposed_gid, unsigned int *matches) { struct smcd_dev *smcd; @@ -2118,7 +2116,11 @@ static void smc_check_ism_v2_match(struct smc_init_info *ini, continue; if (smc_ism_get_chid(smcd) == proposed_chid && !smc_ism_cantalk(proposed_gid, ISM_RESERVED_VLANID, smcd)) { - ini->ism_peer_gid[*matches] = proposed_gid; + ini->ism_peer_gid[*matches].gid = proposed_gid->gid; + if (__smc_ism_is_virtual(proposed_chid)) + ini->ism_peer_gid[*matches].gid_ext = + proposed_gid->gid_ext; + /* non-virtual ISM's peer gid_ext remains 0. */ ini->ism_dev[*matches] = smcd; (*matches)++; break; @@ -2140,9 +2142,11 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, struct smc_clc_v2_extension *smc_v2_ext; struct smc_clc_msg_smcd *pclc_smcd; unsigned int matches = 0; + struct smcd_gid smcd_gid; u8 smcd_version; u8 *eid = NULL; int i, rc; + u16 chid; if (!(ini->smcd_version & SMC_V2) || !smcd_indicated(ini->smc_type_v2)) goto not_found; @@ -2152,18 +2156,35 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, smcd_v2_ext = smc_get_clc_smcd_v2_ext(smc_v2_ext); mutex_lock(&smcd_dev_list.mutex); - if (pclc_smcd->ism.chid) + if (pclc_smcd->ism.chid) { /* check for ISM device matching proposed native ISM device */ + smcd_gid.gid = ntohll(pclc_smcd->ism.gid); + smcd_gid.gid_ext = 0; smc_check_ism_v2_match(ini, ntohs(pclc_smcd->ism.chid), - ntohll(pclc_smcd->ism.gid), &matches); - for (i = 1; i <= smc_v2_ext->hdr.ism_gid_cnt; i++) { + &smcd_gid, &matches); + } + for (i = 0; i < smc_v2_ext->hdr.ism_gid_cnt; i++) { /* check for ISM devices matching proposed non-native ISM * devices */ - smc_check_ism_v2_match(ini, - ntohs(smcd_v2_ext->gidchid[i - 1].chid), - ntohll(smcd_v2_ext->gidchid[i - 1].gid), - &matches); + smcd_gid.gid = ntohll(smcd_v2_ext->gidchid[i].gid); + smcd_gid.gid_ext = 0; + chid = ntohs(smcd_v2_ext->gidchid[i].chid); + if (__smc_ism_is_virtual(chid)) { + if ((i + 1) == smc_v2_ext->hdr.ism_gid_cnt || + chid != ntohs(smcd_v2_ext->gidchid[i + 1].chid)) + /* each virtual ISM device takes two GID-CHID + * entries and CHID of the second entry repeats + * that of the first entry. + * + * So check if the next GID-CHID entry exists + * and both two entries' CHIDs are the same. + */ + continue; + smcd_gid.gid_ext = + ntohll(smcd_v2_ext->gidchid[++i].gid); + } + smc_check_ism_v2_match(ini, chid, &smcd_gid, &matches); } mutex_unlock(&smcd_dev_list.mutex); @@ -2212,7 +2233,8 @@ static void smc_find_ism_v1_device_serv(struct smc_sock *new_smc, if (!(ini->smcd_version & SMC_V1) || !smcd_indicated(ini->smc_type_v1)) goto not_found; ini->is_smcd = true; /* prepare ISM check */ - ini->ism_peer_gid[0] = ntohll(pclc_smcd->ism.gid); + ini->ism_peer_gid[0].gid = ntohll(pclc_smcd->ism.gid); + ini->ism_peer_gid[0].gid_ext = 0; rc = smc_find_ism_device(new_smc, ini); if (rc) goto not_found; diff --git a/net/smc/smc.h b/net/smc/smc.h index cd51261b7d9e..df64efd2dee8 100644 --- a/net/smc/smc.h +++ b/net/smc/smc.h @@ -29,9 +29,6 @@ #define SMCPROTO_SMC 0 /* SMC protocol, IPv4 */ #define SMCPROTO_SMC6 1 /* SMC protocol, IPv6 */ -#define SMC_MAX_ISM_DEVS 8 /* max # of proposed non-native ISM - * devices - */ #define SMC_AUTOCORKING_DEFAULT_SIZE 0x10000 /* 64K by default */ extern struct proto smc_proto; @@ -58,6 +55,13 @@ enum smc_state { /* possible states of an SMC socket */ SMC_PROCESSABORT = 27, }; +enum smc_supplemental_features { + SMC_SPF_VIRT_ISM_DEV = 0, +}; + +#define SMC_FEATURE_MASK \ + (BIT(SMC_SPF_VIRT_ISM_DEV)) + struct smc_link_group; struct smc_wr_rx_hdr { /* common prefix part of LLC and CDC to demultiplex */ diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 95e19aa3e769..9a13709bea1c 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -155,10 +155,12 @@ static int smc_clc_ueid_remove(char *ueid) rc = 0; } } +#if IS_ENABLED(CONFIG_S390) if (!rc && !smc_clc_eid_table.ueid_cnt) { smc_clc_eid_table.seid_enabled = 1; rc = -EAGAIN; /* indicate success and enabling of seid */ } +#endif write_unlock(&smc_clc_eid_table.lock); return rc; } @@ -273,22 +275,30 @@ err: int smc_nl_enable_seid(struct sk_buff *skb, struct genl_info *info) { +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); smc_clc_eid_table.seid_enabled = 1; write_unlock(&smc_clc_eid_table.lock); return 0; +#else + return -EOPNOTSUPP; +#endif } int smc_nl_disable_seid(struct sk_buff *skb, struct genl_info *info) { int rc = 0; +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); if (!smc_clc_eid_table.ueid_cnt) rc = -ENOENT; else smc_clc_eid_table.seid_enabled = 0; write_unlock(&smc_clc_eid_table.lock); +#else + rc = -EOPNOTSUPP; +#endif return rc; } @@ -377,9 +387,9 @@ static bool smc_clc_msg_prop_valid(struct smc_clc_msg_proposal *pclc) /* check arriving CLC accept or confirm */ static bool -smc_clc_msg_acc_conf_valid(struct smc_clc_msg_accept_confirm_v2 *clc_v2) +smc_clc_msg_acc_conf_valid(struct smc_clc_msg_accept_confirm *clc) { - struct smc_clc_msg_hdr *hdr = &clc_v2->hdr; + struct smc_clc_msg_hdr *hdr = &clc->hdr; if (hdr->typev1 != SMC_TYPE_R && hdr->typev1 != SMC_TYPE_D) return false; @@ -418,15 +428,16 @@ smc_clc_msg_decl_valid(struct smc_clc_msg_decline *dclc) return true; } -static int smc_clc_fill_fce(struct smc_clc_first_contact_ext_v2x *fce, - struct smc_init_info *ini) +static int smc_clc_fill_fce_v2x(struct smc_clc_first_contact_ext_v2x *fce_v2x, + struct smc_init_info *ini) { - int ret = sizeof(*fce); + int ret = sizeof(*fce_v2x); - memset(fce, 0, sizeof(*fce)); - fce->fce_v2_base.os_type = SMC_CLC_OS_LINUX; - fce->fce_v2_base.release = ini->release_nr; - memcpy(fce->fce_v2_base.hostname, smc_hostname, sizeof(smc_hostname)); + memset(fce_v2x, 0, sizeof(*fce_v2x)); + fce_v2x->fce_v2_base.os_type = SMC_CLC_OS_LINUX; + fce_v2x->fce_v2_base.release = ini->release_nr; + memcpy(fce_v2x->fce_v2_base.hostname, + smc_hostname, sizeof(smc_hostname)); if (ini->is_smcd && ini->release_nr < SMC_RELEASE_1) { ret = sizeof(struct smc_clc_first_contact_ext); goto out; @@ -434,9 +445,10 @@ static int smc_clc_fill_fce(struct smc_clc_first_contact_ext_v2x *fce, if (ini->release_nr >= SMC_RELEASE_1) { if (!ini->is_smcd) { - fce->max_conns = ini->max_conns; - fce->max_links = ini->max_links; + fce_v2x->max_conns = ini->max_conns; + fce_v2x->max_links = ini->max_links; } + fce_v2x->feature_mask = htons(ini->feature_mask); } out: @@ -448,7 +460,7 @@ out: */ static bool smc_clc_msg_hdr_valid(struct smc_clc_msg_hdr *clcm, bool check_trl) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2; + struct smc_clc_msg_accept_confirm *clc; struct smc_clc_msg_proposal *pclc; struct smc_clc_msg_decline *dclc; struct smc_clc_msg_trail *trl; @@ -466,12 +478,11 @@ static bool smc_clc_msg_hdr_valid(struct smc_clc_msg_hdr *clcm, bool check_trl) break; case SMC_CLC_ACCEPT: case SMC_CLC_CONFIRM: - clc_v2 = (struct smc_clc_msg_accept_confirm_v2 *)clcm; - if (!smc_clc_msg_acc_conf_valid(clc_v2)) + clc = (struct smc_clc_msg_accept_confirm *)clcm; + if (!smc_clc_msg_acc_conf_valid(clc)) return false; trl = (struct smc_clc_msg_trail *) - ((u8 *)clc_v2 + ntohs(clc_v2->hdr.length) - - sizeof(*trl)); + ((u8 *)clc + ntohs(clc->hdr.length) - sizeof(*trl)); break; case SMC_CLC_DECLINE: dclc = (struct smc_clc_msg_decline *)clcm; @@ -882,11 +893,13 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) ETH_ALEN); } if (smcd_indicated(ini->smc_type_v1)) { + struct smcd_gid smcd_gid; + /* add SMC-D specifics */ if (ini->ism_dev[0]) { smcd = ini->ism_dev[0]; - pclc_smcd->ism.gid = - htonll(smcd->ops->get_local_gid(smcd)); + smcd->ops->get_local_gid(smcd, &smcd_gid); + pclc_smcd->ism.gid = htonll(smcd_gid.gid); pclc_smcd->ism.chid = htons(smc_ism_get_chid(ini->ism_dev[0])); } @@ -907,6 +920,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) pclc_smcd->v2_ext_offset = htons(v2_ext_offset); plen += sizeof(*v2_ext); + v2_ext->feature_mask = htons(SMC_FEATURE_MASK); read_lock(&smc_clc_eid_table.lock); v2_ext->hdr.eid_cnt = smc_clc_eid_table.ueid_cnt; plen += smc_clc_eid_table.ueid_cnt * SMC_MAX_EID_LEN; @@ -918,10 +932,11 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) read_unlock(&smc_clc_eid_table.lock); } if (smcd_indicated(ini->smc_type_v2)) { + struct smcd_gid smcd_gid; u8 *eid = NULL; + int entry = 0; v2_ext->hdr.flag.seid = smc_clc_eid_table.seid_enabled; - v2_ext->hdr.ism_gid_cnt = ini->ism_offered_cnt; v2_ext->hdr.smcd_v2_ext_offset = htons(sizeof(*v2_ext) - offsetofend(struct smc_clnt_opts_area_hdr, smcd_v2_ext_offset) + @@ -933,14 +948,26 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) if (ini->ism_offered_cnt) { for (i = 1; i <= ini->ism_offered_cnt; i++) { smcd = ini->ism_dev[i]; - gidchids[i - 1].gid = - htonll(smcd->ops->get_local_gid(smcd)); - gidchids[i - 1].chid = + smcd->ops->get_local_gid(smcd, &smcd_gid); + gidchids[entry].chid = htons(smc_ism_get_chid(ini->ism_dev[i])); + gidchids[entry].gid = htonll(smcd_gid.gid); + if (smc_ism_is_virtual(smcd)) { + /* a virtual ISM device takes two + * entries. CHID of the second entry + * repeats that of the first entry. + */ + gidchids[entry + 1].chid = + gidchids[entry].chid; + gidchids[entry + 1].gid = + htonll(smcd_gid.gid_ext); + entry++; + } + entry++; } - plen += ini->ism_offered_cnt * - sizeof(struct smc_clc_smcd_gid_chid); + plen += entry * sizeof(struct smc_clc_smcd_gid_chid); } + v2_ext->hdr.ism_gid_cnt = entry; } if (smcr_indicated(ini->smc_type_v2)) { memcpy(v2_ext->roce, ini->smcrv2.ib_gid_v2, SMC_GID_SIZE); @@ -976,7 +1003,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) vec[i++].iov_len = sizeof(*smcd_v2_ext); if (ini->ism_offered_cnt) { vec[i].iov_base = gidchids; - vec[i++].iov_len = ini->ism_offered_cnt * + vec[i++].iov_len = v2_ext->hdr.ism_gid_cnt * sizeof(struct smc_clc_smcd_gid_chid); } } @@ -997,109 +1024,143 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) return reason_code; } +static void +smcd_clc_prep_confirm_accept(struct smc_connection *conn, + struct smc_clc_msg_accept_confirm *clc, + int first_contact, u8 version, + u8 *eid, struct smc_init_info *ini, + int *fce_len, + struct smc_clc_first_contact_ext_v2x *fce_v2x, + struct smc_clc_msg_trail *trl) +{ + struct smcd_dev *smcd = conn->lgr->smcd; + struct smcd_gid smcd_gid; + u16 chid; + int len; + + /* SMC-D specific settings */ + memcpy(clc->hdr.eyecatcher, SMCD_EYECATCHER, + sizeof(SMCD_EYECATCHER)); + smcd->ops->get_local_gid(smcd, &smcd_gid); + clc->hdr.typev1 = SMC_TYPE_D; + clc->d0.gid = htonll(smcd_gid.gid); + clc->d0.token = htonll(conn->rmb_desc->token); + clc->d0.dmbe_size = conn->rmbe_size_comp; + clc->d0.dmbe_idx = 0; + memcpy(&clc->d0.linkid, conn->lgr->id, SMC_LGR_ID_SIZE); + if (version == SMC_V1) { + clc->hdr.length = htons(SMCD_CLC_ACCEPT_CONFIRM_LEN); + } else { + chid = smc_ism_get_chid(smcd); + clc->d1.chid = htons(chid); + if (eid && eid[0]) + memcpy(clc->d1.eid, eid, SMC_MAX_EID_LEN); + if (__smc_ism_is_virtual(chid)) + clc->d1.gid_ext = htonll(smcd_gid.gid_ext); + len = SMCD_CLC_ACCEPT_CONFIRM_LEN_V2; + if (first_contact) { + *fce_len = smc_clc_fill_fce_v2x(fce_v2x, ini); + len += *fce_len; + } + clc->hdr.length = htons(len); + } + memcpy(trl->eyecatcher, SMCD_EYECATCHER, + sizeof(SMCD_EYECATCHER)); +} + +static void +smcr_clc_prep_confirm_accept(struct smc_connection *conn, + struct smc_clc_msg_accept_confirm *clc, + int first_contact, u8 version, + u8 *eid, struct smc_init_info *ini, + int *fce_len, + struct smc_clc_first_contact_ext_v2x *fce_v2x, + struct smc_clc_fce_gid_ext *gle, + struct smc_clc_msg_trail *trl) +{ + struct smc_link *link = conn->lnk; + int len; + + /* SMC-R specific settings */ + memcpy(clc->hdr.eyecatcher, SMC_EYECATCHER, + sizeof(SMC_EYECATCHER)); + clc->hdr.typev1 = SMC_TYPE_R; + memcpy(clc->r0.lcl.id_for_peer, local_systemid, + sizeof(local_systemid)); + memcpy(&clc->r0.lcl.gid, link->gid, SMC_GID_SIZE); + memcpy(&clc->r0.lcl.mac, &link->smcibdev->mac[link->ibport - 1], + ETH_ALEN); + hton24(clc->r0.qpn, link->roce_qp->qp_num); + clc->r0.rmb_rkey = + htonl(conn->rmb_desc->mr[link->link_idx]->rkey); + clc->r0.rmbe_idx = 1; /* for now: 1 RMB = 1 RMBE */ + clc->r0.rmbe_alert_token = htonl(conn->alert_token_local); + switch (clc->hdr.type) { + case SMC_CLC_ACCEPT: + clc->r0.qp_mtu = link->path_mtu; + break; + case SMC_CLC_CONFIRM: + clc->r0.qp_mtu = min(link->path_mtu, link->peer_mtu); + break; + } + clc->r0.rmbe_size = conn->rmbe_size_comp; + clc->r0.rmb_dma_addr = conn->rmb_desc->is_vm ? + cpu_to_be64((uintptr_t)conn->rmb_desc->cpu_addr) : + cpu_to_be64((u64)sg_dma_address + (conn->rmb_desc->sgt[link->link_idx].sgl)); + hton24(clc->r0.psn, link->psn_initial); + if (version == SMC_V1) { + clc->hdr.length = htons(SMCR_CLC_ACCEPT_CONFIRM_LEN); + } else { + if (eid && eid[0]) + memcpy(clc->r1.eid, eid, SMC_MAX_EID_LEN); + len = SMCR_CLC_ACCEPT_CONFIRM_LEN_V2; + if (first_contact) { + *fce_len = smc_clc_fill_fce_v2x(fce_v2x, ini); + len += *fce_len; + fce_v2x->fce_v2_base.v2_direct = + !link->lgr->uses_gateway; + if (clc->hdr.type == SMC_CLC_CONFIRM) { + memset(gle, 0, sizeof(*gle)); + gle->gid_cnt = ini->smcrv2.gidlist.len; + len += sizeof(*gle); + len += gle->gid_cnt * sizeof(gle->gid[0]); + } + } + clc->hdr.length = htons(len); + } + memcpy(trl->eyecatcher, SMC_EYECATCHER, sizeof(SMC_EYECATCHER)); +} + /* build and send CLC CONFIRM / ACCEPT message */ static int smc_clc_send_confirm_accept(struct smc_sock *smc, - struct smc_clc_msg_accept_confirm_v2 *clc_v2, + struct smc_clc_msg_accept_confirm *clc, int first_contact, u8 version, u8 *eid, struct smc_init_info *ini) { + struct smc_clc_first_contact_ext_v2x fce_v2x; struct smc_connection *conn = &smc->conn; - struct smc_clc_first_contact_ext_v2x fce; - struct smcd_dev *smcd = conn->lgr->smcd; - struct smc_clc_msg_accept_confirm *clc; struct smc_clc_fce_gid_ext gle; struct smc_clc_msg_trail trl; - int i, len, fce_len; + int i, fce_len; struct kvec vec[5]; struct msghdr msg; /* send SMC Confirm CLC msg */ - clc = (struct smc_clc_msg_accept_confirm *)clc_v2; clc->hdr.version = version; /* SMC version */ if (first_contact) clc->hdr.typev2 |= SMC_FIRST_CONTACT_MASK; - if (conn->lgr->is_smcd) { - /* SMC-D specific settings */ - memcpy(clc->hdr.eyecatcher, SMCD_EYECATCHER, - sizeof(SMCD_EYECATCHER)); - clc->hdr.typev1 = SMC_TYPE_D; - clc->d0.gid = htonll(smcd->ops->get_local_gid(smcd)); - clc->d0.token = htonll(conn->rmb_desc->token); - clc->d0.dmbe_size = conn->rmbe_size_comp; - clc->d0.dmbe_idx = 0; - memcpy(&clc->d0.linkid, conn->lgr->id, SMC_LGR_ID_SIZE); - if (version == SMC_V1) { - clc->hdr.length = htons(SMCD_CLC_ACCEPT_CONFIRM_LEN); - } else { - clc_v2->d1.chid = htons(smc_ism_get_chid(smcd)); - if (eid && eid[0]) - memcpy(clc_v2->d1.eid, eid, SMC_MAX_EID_LEN); - len = SMCD_CLC_ACCEPT_CONFIRM_LEN_V2; - if (first_contact) { - fce_len = smc_clc_fill_fce(&fce, ini); - len += fce_len; - } - clc_v2->hdr.length = htons(len); - } - memcpy(trl.eyecatcher, SMCD_EYECATCHER, - sizeof(SMCD_EYECATCHER)); - } else { - struct smc_link *link = conn->lnk; - - /* SMC-R specific settings */ - memcpy(clc->hdr.eyecatcher, SMC_EYECATCHER, - sizeof(SMC_EYECATCHER)); - clc->hdr.typev1 = SMC_TYPE_R; - clc->hdr.length = htons(SMCR_CLC_ACCEPT_CONFIRM_LEN); - memcpy(clc->r0.lcl.id_for_peer, local_systemid, - sizeof(local_systemid)); - memcpy(&clc->r0.lcl.gid, link->gid, SMC_GID_SIZE); - memcpy(&clc->r0.lcl.mac, &link->smcibdev->mac[link->ibport - 1], - ETH_ALEN); - hton24(clc->r0.qpn, link->roce_qp->qp_num); - clc->r0.rmb_rkey = - htonl(conn->rmb_desc->mr[link->link_idx]->rkey); - clc->r0.rmbe_idx = 1; /* for now: 1 RMB = 1 RMBE */ - clc->r0.rmbe_alert_token = htonl(conn->alert_token_local); - switch (clc->hdr.type) { - case SMC_CLC_ACCEPT: - clc->r0.qp_mtu = link->path_mtu; - break; - case SMC_CLC_CONFIRM: - clc->r0.qp_mtu = min(link->path_mtu, link->peer_mtu); - break; - } - clc->r0.rmbe_size = conn->rmbe_size_comp; - clc->r0.rmb_dma_addr = conn->rmb_desc->is_vm ? - cpu_to_be64((uintptr_t)conn->rmb_desc->cpu_addr) : - cpu_to_be64((u64)sg_dma_address - (conn->rmb_desc->sgt[link->link_idx].sgl)); - hton24(clc->r0.psn, link->psn_initial); - if (version == SMC_V1) { - clc->hdr.length = htons(SMCR_CLC_ACCEPT_CONFIRM_LEN); - } else { - if (eid && eid[0]) - memcpy(clc_v2->r1.eid, eid, SMC_MAX_EID_LEN); - len = SMCR_CLC_ACCEPT_CONFIRM_LEN_V2; - if (first_contact) { - fce_len = smc_clc_fill_fce(&fce, ini); - len += fce_len; - fce.fce_v2_base.v2_direct = !link->lgr->uses_gateway; - if (clc->hdr.type == SMC_CLC_CONFIRM) { - memset(&gle, 0, sizeof(gle)); - gle.gid_cnt = ini->smcrv2.gidlist.len; - len += sizeof(gle); - len += gle.gid_cnt * sizeof(gle.gid[0]); - } - } - clc_v2->hdr.length = htons(len); - } - memcpy(trl.eyecatcher, SMC_EYECATCHER, sizeof(SMC_EYECATCHER)); - } - + if (conn->lgr->is_smcd) + smcd_clc_prep_confirm_accept(conn, clc, first_contact, + version, eid, ini, &fce_len, + &fce_v2x, &trl); + else + smcr_clc_prep_confirm_accept(conn, clc, first_contact, + version, eid, ini, &fce_len, + &fce_v2x, &gle, &trl); memset(&msg, 0, sizeof(msg)); i = 0; - vec[i].iov_base = clc_v2; + vec[i].iov_base = clc; if (version > SMC_V1) vec[i++].iov_len = (clc->hdr.typev1 == SMC_TYPE_D ? SMCD_CLC_ACCEPT_CONFIRM_LEN_V2 : @@ -1111,7 +1172,7 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc, SMCR_CLC_ACCEPT_CONFIRM_LEN) - sizeof(trl); if (version > SMC_V1 && first_contact) { - vec[i].iov_base = &fce; + vec[i].iov_base = &fce_v2x; vec[i++].iov_len = fce_len; if (!conn->lgr->is_smcd) { if (clc->hdr.type == SMC_CLC_CONFIRM) { @@ -1133,16 +1194,16 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc, int smc_clc_send_confirm(struct smc_sock *smc, bool clnt_first_contact, u8 version, u8 *eid, struct smc_init_info *ini) { - struct smc_clc_msg_accept_confirm_v2 cclc_v2; + struct smc_clc_msg_accept_confirm cclc; int reason_code = 0; int len; /* send SMC Confirm CLC msg */ - memset(&cclc_v2, 0, sizeof(cclc_v2)); - cclc_v2.hdr.type = SMC_CLC_CONFIRM; - len = smc_clc_send_confirm_accept(smc, &cclc_v2, clnt_first_contact, + memset(&cclc, 0, sizeof(cclc)); + cclc.hdr.type = SMC_CLC_CONFIRM; + len = smc_clc_send_confirm_accept(smc, &cclc, clnt_first_contact, version, eid, ini); - if (len < ntohs(cclc_v2.hdr.length)) { + if (len < ntohs(cclc.hdr.length)) { if (len >= 0) { reason_code = -ENETUNREACH; smc->sk.sk_err = -reason_code; @@ -1158,14 +1219,14 @@ int smc_clc_send_confirm(struct smc_sock *smc, bool clnt_first_contact, int smc_clc_send_accept(struct smc_sock *new_smc, bool srv_first_contact, u8 version, u8 *negotiated_eid, struct smc_init_info *ini) { - struct smc_clc_msg_accept_confirm_v2 aclc_v2; + struct smc_clc_msg_accept_confirm aclc; int len; - memset(&aclc_v2, 0, sizeof(aclc_v2)); - aclc_v2.hdr.type = SMC_CLC_ACCEPT; - len = smc_clc_send_confirm_accept(new_smc, &aclc_v2, srv_first_contact, + memset(&aclc, 0, sizeof(aclc)); + aclc.hdr.type = SMC_CLC_ACCEPT; + len = smc_clc_send_confirm_accept(new_smc, &aclc, srv_first_contact, version, negotiated_eid, ini); - if (len < ntohs(aclc_v2.hdr.length)) + if (len < ntohs(aclc.hdr.length)) len = len >= 0 ? -EPROTO : -new_smc->clcsock->sk->sk_err; return len > 0 ? 0 : len; @@ -1180,6 +1241,7 @@ int smc_clc_srv_v2x_features_validate(struct smc_sock *smc, ini->max_conns = SMC_CONN_PER_LGR_MAX; ini->max_links = SMC_LINKS_ADD_LNK_MAX; + ini->feature_mask = SMC_FEATURE_MASK; if ((!(ini->smcd_version & SMC_V2) && !(ini->smcr_version & SMC_V2)) || ini->release_nr < SMC_RELEASE_1) @@ -1223,6 +1285,8 @@ int smc_clc_clnt_v2x_features_validate(struct smc_clc_first_contact_ext *fce, return SMC_CLC_DECL_MAXLINKERR; ini->max_links = fce_v2x->max_links; } + /* common supplemental features of server and client */ + ini->feature_mask = ntohs(fce_v2x->feature_mask) & SMC_FEATURE_MASK; return 0; } @@ -1230,10 +1294,8 @@ int smc_clc_clnt_v2x_features_validate(struct smc_clc_first_contact_ext *fce, int smc_clc_v2x_features_confirm_check(struct smc_clc_msg_accept_confirm *cclc, struct smc_init_info *ini) { - struct smc_clc_msg_accept_confirm_v2 *clc_v2 = - (struct smc_clc_msg_accept_confirm_v2 *)cclc; struct smc_clc_first_contact_ext *fce = - smc_get_clc_first_contact_ext(clc_v2, ini->is_smcd); + smc_get_clc_first_contact_ext(cclc, ini->is_smcd); struct smc_clc_first_contact_ext_v2x *fce_v2x = (struct smc_clc_first_contact_ext_v2x *)fce; @@ -1253,6 +1315,8 @@ int smc_clc_v2x_features_confirm_check(struct smc_clc_msg_accept_confirm *cclc, if (fce_v2x->max_links != ini->max_links) return SMC_CLC_DECL_MAXLINKERR; } + /* common supplemental features returned by client */ + ini->feature_mask = ntohs(fce_v2x->feature_mask); return 0; } @@ -1274,7 +1338,11 @@ void __init smc_clc_init(void) INIT_LIST_HEAD(&smc_clc_eid_table.list); rwlock_init(&smc_clc_eid_table.lock); smc_clc_eid_table.ueid_cnt = 0; +#if IS_ENABLED(CONFIG_S390) smc_clc_eid_table.seid_enabled = 1; +#else + smc_clc_eid_table.seid_enabled = 0; +#endif } void smc_clc_exit(void) diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h index 1697b84d85be..a9f9bdd26dcd 100644 --- a/net/smc/smc_clc.h +++ b/net/smc/smc_clc.h @@ -138,7 +138,8 @@ struct smc_clc_v2_extension { u8 roce[16]; /* RoCEv2 GID */ u8 max_conns; u8 max_links; - u8 reserved[14]; + __be16 feature_mask; + u8 reserved[12]; u8 user_eids[][SMC_MAX_EID_LEN]; }; @@ -171,6 +172,11 @@ struct smc_clc_msg_proposal { /* clc proposal message sent by Linux */ #define SMC_CLC_MAX_V6_PREFIX 8 #define SMC_CLC_MAX_UEID 8 +#define SMCD_CLC_MAX_V2_GID_ENTRIES 8 /* max # of CHID-GID entries in CLC + * proposal SMC-Dv2 extension. + * each ISM device takes one entry and + * each virtual ISM takes two entries. + */ struct smc_clc_msg_proposal_area { struct smc_clc_msg_proposal pclc_base; @@ -180,7 +186,8 @@ struct smc_clc_msg_proposal_area { struct smc_clc_v2_extension pclc_v2_ext; u8 user_eids[SMC_CLC_MAX_UEID][SMC_MAX_EID_LEN]; struct smc_clc_smcd_v2_extension pclc_smcd_v2_ext; - struct smc_clc_smcd_gid_chid pclc_gidchids[SMC_MAX_ISM_DEVS]; + struct smc_clc_smcd_gid_chid + pclc_gidchids[SMCD_CLC_MAX_V2_GID_ENTRIES]; struct smc_clc_msg_trail pclc_trl; }; @@ -240,9 +247,14 @@ struct smc_clc_first_contact_ext { struct smc_clc_first_contact_ext_v2x { struct smc_clc_first_contact_ext fce_v2_base; - u8 max_conns; /* for SMC-R only */ - u8 max_links; /* for SMC-R only */ - u8 reserved3[2]; + union { + struct { + u8 max_conns; /* for SMC-R only */ + u8 max_links; /* for SMC-R only */ + }; + u8 reserved3[2]; /* for SMC-D only */ + }; + __be16 feature_mask; __be32 vendor_exp_options; u8 reserved4[8]; } __packed; /* format defined in @@ -259,28 +271,21 @@ struct smc_clc_fce_gid_ext { struct smc_clc_msg_accept_confirm { /* clc accept / confirm message */ struct smc_clc_msg_hdr hdr; union { - struct smcr_clc_msg_accept_confirm r0; /* SMC-R */ - struct { /* SMC-D */ - struct smcd_clc_msg_accept_confirm_common d0; - u32 reserved5[3]; - }; - }; -} __packed; /* format defined in RFC7609 */ - -struct smc_clc_msg_accept_confirm_v2 { /* clc accept / confirm message */ - struct smc_clc_msg_hdr hdr; - union { struct { /* SMC-R */ struct smcr_clc_msg_accept_confirm r0; - u8 eid[SMC_MAX_EID_LEN]; - u8 reserved6[8]; - } r1; + struct { /* v2 only */ + u8 eid[SMC_MAX_EID_LEN]; + u8 reserved6[8]; + } __packed r1; + }; struct { /* SMC-D */ struct smcd_clc_msg_accept_confirm_common d0; - __be16 chid; - u8 eid[SMC_MAX_EID_LEN]; - u8 reserved5[8]; - } d1; + struct { /* v2 only, but 12 bytes reserved in v1 */ + __be16 chid; + u8 eid[SMC_MAX_EID_LEN]; + __be64 gid_ext; + } __packed d1; + }; }; }; @@ -389,24 +394,23 @@ smc_get_clc_smcd_v2_ext(struct smc_clc_v2_extension *prop_v2ext) } static inline struct smc_clc_first_contact_ext * -smc_get_clc_first_contact_ext(struct smc_clc_msg_accept_confirm_v2 *clc_v2, +smc_get_clc_first_contact_ext(struct smc_clc_msg_accept_confirm *clc, bool is_smcd) { int clc_v2_len; - if (clc_v2->hdr.version == SMC_V1 || - !(clc_v2->hdr.typev2 & SMC_FIRST_CONTACT_MASK)) + if (clc->hdr.version == SMC_V1 || + !(clc->hdr.typev2 & SMC_FIRST_CONTACT_MASK)) return NULL; if (is_smcd) clc_v2_len = - offsetofend(struct smc_clc_msg_accept_confirm_v2, d1); + offsetofend(struct smc_clc_msg_accept_confirm, d1); else clc_v2_len = - offsetofend(struct smc_clc_msg_accept_confirm_v2, r1); + offsetofend(struct smc_clc_msg_accept_confirm, r1); - return (struct smc_clc_first_contact_ext *)(((u8 *)clc_v2) + - clc_v2_len); + return (struct smc_clc_first_contact_ext *)(((u8 *)clc) + clc_v2_len); } struct smcd_dev; diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index d520ee62c8ec..95cc95458e2d 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -506,6 +506,7 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, { char smc_pnet[SMC_MAX_PNETID_LEN + 1]; struct smcd_dev *smcd = lgr->smcd; + struct smcd_gid smcd_gid; struct nlattr *attrs; void *nlh; @@ -521,13 +522,19 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, if (nla_put_u32(skb, SMC_NLA_LGR_D_ID, *((u32 *)&lgr->id))) goto errattr; + smcd->ops->get_local_gid(smcd, &smcd_gid); if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_GID, - smcd->ops->get_local_gid(smcd), - SMC_NLA_LGR_D_PAD)) + smcd_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; - if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid, + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_EXT_GID, + smcd_gid.gid_ext, SMC_NLA_LGR_D_PAD)) + goto errattr; + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_EXT_GID, + lgr->peer_gid.gid_ext, SMC_NLA_LGR_D_PAD)) + goto errattr; if (nla_put_u8(skb, SMC_NLA_LGR_D_VLAN_ID, lgr->vlan_id)) goto errattr; if (nla_put_u32(skb, SMC_NLA_LGR_D_CONNS_NUM, lgr->conns_num)) @@ -876,7 +883,10 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini) /* SMC-D specific settings */ smcd = ini->ism_dev[ini->ism_selected]; get_device(smcd->ops->get_dev(smcd)); - lgr->peer_gid = ini->ism_peer_gid[ini->ism_selected]; + lgr->peer_gid.gid = + ini->ism_peer_gid[ini->ism_selected].gid; + lgr->peer_gid.gid_ext = + ini->ism_peer_gid[ini->ism_selected].gid_ext; lgr->smcd = ini->ism_dev[ini->ism_selected]; lgr_list = &ini->ism_dev[ini->ism_selected]->lgr_list; lgr_lock = &lgr->smcd->lgr_lock; @@ -1514,7 +1524,8 @@ void smc_lgr_terminate_sched(struct smc_link_group *lgr) } /* Called when peer lgr shutdown (regularly or abnormally) is received */ -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, + unsigned short vlan) { struct smc_link_group *lgr, *l; LIST_HEAD(lgr_free_list); @@ -1522,9 +1533,12 @@ void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) /* run common cleanup function and build free list */ spin_lock_bh(&dev->lgr_lock); list_for_each_entry_safe(lgr, l, &dev->lgr_list, list) { - if ((!peer_gid || lgr->peer_gid == peer_gid) && + if ((!peer_gid->gid || + (lgr->peer_gid.gid == peer_gid->gid && + !smc_ism_is_virtual(dev) ? 1 : + lgr->peer_gid.gid_ext == peer_gid->gid_ext)) && (vlan == VLAN_VID_MASK || lgr->vlan_id == vlan)) { - if (peer_gid) /* peer triggered termination */ + if (peer_gid->gid) /* peer triggered termination */ lgr->peer_shutdown = 1; list_move(&lgr->list, &lgr_free_list); lgr->freeing = 1; @@ -1860,9 +1874,12 @@ static bool smcr_lgr_match(struct smc_link_group *lgr, u8 smcr_version, } static bool smcd_lgr_match(struct smc_link_group *lgr, - struct smcd_dev *smcismdev, u64 peer_gid) + struct smcd_dev *smcismdev, + struct smcd_gid *peer_gid) { - return lgr->peer_gid == peer_gid && lgr->smcd == smcismdev; + return lgr->peer_gid.gid == peer_gid->gid && lgr->smcd == smcismdev && + smc_ism_is_virtual(smcismdev) ? + (lgr->peer_gid.gid_ext == peer_gid->gid_ext) : 1; } /* create a new SMC connection (and a new link group if necessary) */ @@ -1892,7 +1909,7 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini) write_lock_bh(&lgr->conns_lock); if ((ini->is_smcd ? smcd_lgr_match(lgr, ini->ism_dev[ini->ism_selected], - ini->ism_peer_gid[ini->ism_selected]) : + &ini->ism_peer_gid[ini->ism_selected]) : smcr_lgr_match(lgr, ini->smcr_version, ini->peer_systemid, ini->peer_gid, ini->peer_mac, role, diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 120027d40469..1f175376037b 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -17,9 +17,11 @@ #include <linux/pci.h> #include <rdma/ib_verbs.h> #include <net/genetlink.h> +#include <net/smc.h> #include "smc.h" #include "smc_ib.h" +#include "smc_clc.h" #define SMC_RMBS_PER_LGR_MAX 255 /* max. # of RMBs per link group */ #define SMC_CONN_PER_LGR_MIN 16 /* min. # of connections per link group */ @@ -355,7 +357,7 @@ struct smc_link_group { /* max links can be added in lgr */ }; struct { /* SMC-D */ - u64 peer_gid; + struct smcd_gid peer_gid; /* Peer GID (remote) */ struct smcd_dev *smcd; /* ISM device for VLAN reg. */ @@ -392,6 +394,11 @@ struct smc_init_info_smcrv2 { struct smc_gidlist gidlist; }; +#define SMC_MAX_V2_ISM_DEVS SMCD_CLC_MAX_V2_GID_ENTRIES + /* max # of proposed non-native ISM devices, + * which can't exceed the max # of CHID-GID + * entries in CLC proposal SMC-Dv2 extension. + */ struct smc_init_info { u8 is_smcd; u8 smc_type_v1; @@ -401,6 +408,7 @@ struct smc_init_info { u8 max_links; u8 first_contact_peer; u8 first_contact_local; + u16 feature_mask; unsigned short vlan_id; u32 rc; u8 negotiated_eid[SMC_MAX_EID_LEN]; @@ -416,9 +424,9 @@ struct smc_init_info { u32 ib_clcqpn; struct smc_init_info_smcrv2 smcrv2; /* SMC-D */ - u64 ism_peer_gid[SMC_MAX_ISM_DEVS + 1]; - struct smcd_dev *ism_dev[SMC_MAX_ISM_DEVS + 1]; - u16 ism_chid[SMC_MAX_ISM_DEVS + 1]; + struct smcd_gid ism_peer_gid[SMC_MAX_V2_ISM_DEVS + 1]; + struct smcd_dev *ism_dev[SMC_MAX_V2_ISM_DEVS + 1]; + u16 ism_chid[SMC_MAX_V2_ISM_DEVS + 1]; u8 ism_offered_cnt; /* # of ISM devices offered */ u8 ism_selected; /* index of selected ISM dev*/ u8 smcd_version; @@ -544,7 +552,7 @@ void smc_lgr_hold(struct smc_link_group *lgr); void smc_lgr_put(struct smc_link_group *lgr); void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport); void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport); -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, unsigned short vlan); void smc_smcd_terminate_all(struct smcd_dev *dev); void smc_smcr_terminate_all(struct smc_ib_device *smcibdev); diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c index a584613aca12..3fbe14e09ad8 100644 --- a/net/smc/smc_diag.c +++ b/net/smc/smc_diag.c @@ -21,6 +21,7 @@ #include "smc.h" #include "smc_core.h" +#include "smc_ism.h" struct smc_diag_dump_ctx { int pos[2]; @@ -168,12 +169,16 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb, struct smc_connection *conn = &smc->conn; struct smcd_diag_dmbinfo dinfo; struct smcd_dev *smcd = conn->lgr->smcd; + struct smcd_gid smcd_gid; memset(&dinfo, 0, sizeof(dinfo)); dinfo.linkid = *((u32 *)conn->lgr->id); - dinfo.peer_gid = conn->lgr->peer_gid; - dinfo.my_gid = smcd->ops->get_local_gid(smcd); + dinfo.peer_gid = conn->lgr->peer_gid.gid; + dinfo.peer_gid_ext = conn->lgr->peer_gid.gid_ext; + smcd->ops->get_local_gid(smcd, &smcd_gid); + dinfo.my_gid = smcd_gid.gid; + dinfo.my_gid_ext = smcd_gid.gid_ext; dinfo.token = conn->rmb_desc->token; dinfo.peer_token = conn->peer_token; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index fbee2493091f..ac88de2a06a0 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -43,8 +43,30 @@ static struct ism_client smc_ism_client = { }; #endif +static void smc_ism_create_system_eid(void) +{ + struct smc_ism_seid *seid = + (struct smc_ism_seid *)smc_ism_v2_system_eid; +#if IS_ENABLED(CONFIG_S390) + struct cpuid id; + u16 ident_tail; + char tmp[5]; + + memcpy(seid->seid_string, "IBM-SYSZ-ISMSEID00000000", 24); + get_cpu_id(&id); + ident_tail = (u16)(id.ident & SMC_ISM_IDENT_MASK); + snprintf(tmp, 5, "%04X", ident_tail); + memcpy(seid->serial_number, tmp, 4); + snprintf(tmp, 5, "%04X", id.machine); + memcpy(seid->type, tmp, 4); +#else + memset(seid, 0, SMC_MAX_EID_LEN); +#endif +} + /* Test if an ISM communication is possible - same CPC */ -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *smcd) +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *smcd) { return smcd->ops->query_remote_gid(smcd, peer_gid, vlan_id ? 1 : 0, vlan_id); @@ -208,7 +230,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.dmb_len = dmb_len; dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; - dmb.rgid = lgr->peer_gid; + dmb.rgid = lgr->peer_gid.gid; rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, &smc_ism_client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; @@ -340,18 +362,20 @@ union smcd_sw_event_info { static void smcd_handle_sw_event(struct smc_ism_event_work *wrk) { + struct smcd_gid peer_gid = { .gid = wrk->event.tok, + .gid_ext = 0 }; union smcd_sw_event_info ev_info; ev_info.info = wrk->event.info; switch (wrk->event.code) { case ISM_EVENT_CODE_SHUTDOWN: /* Peer shut down DMBs */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, ev_info.vlan_id); + smc_smcd_terminate(wrk->smcd, &peer_gid, ev_info.vlan_id); break; case ISM_EVENT_CODE_TESTLINK: /* Activity timer */ if (ev_info.code == ISM_EVENT_REQUEST) { ev_info.code = ISM_EVENT_RESPONSE; wrk->smcd->ops->signal_event(wrk->smcd, - wrk->event.tok, + &peer_gid, ISM_EVENT_REQUEST_IR, ISM_EVENT_CODE_TESTLINK, ev_info.info); @@ -365,10 +389,12 @@ static void smc_ism_event_work(struct work_struct *work) { struct smc_ism_event_work *wrk = container_of(work, struct smc_ism_event_work, work); + struct smcd_gid smcd_gid = { .gid = wrk->event.tok, + .gid_ext = 0 }; switch (wrk->event.type) { case ISM_EVENT_GID: /* GID event, token is peer GID */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, VLAN_VID_MASK); + smc_smcd_terminate(wrk->smcd, &smcd_gid, VLAN_VID_MASK); break; case ISM_EVENT_DMB: break; @@ -426,14 +452,8 @@ static void smcd_register_dev(struct ism_dev *ism) mutex_lock(&smcd_dev_list.mutex); if (list_empty(&smcd_dev_list.list)) { - u8 *system_eid = NULL; - - system_eid = smcd->ops->get_system_eid(); - if (smcd->ops->supports_v2()) { + if (smcd->ops->supports_v2()) smc_ism_v2_capable = true; - memcpy(smc_ism_v2_system_eid, system_eid, - SMC_MAX_EID_LEN); - } } /* sort list: devices without pnetid before devices with pnetid */ if (smcd->pnetid[0]) @@ -525,7 +545,7 @@ int smc_ism_signal_shutdown(struct smc_link_group *lgr) memcpy(ev_info.uid, lgr->id, SMC_LGR_ID_SIZE); ev_info.vlan_id = lgr->vlan_id; ev_info.code = ISM_EVENT_REQUEST; - rc = lgr->smcd->ops->signal_event(lgr->smcd, lgr->peer_gid, + rc = lgr->smcd->ops->signal_event(lgr->smcd, &lgr->peer_gid, ISM_EVENT_REQUEST_IR, ISM_EVENT_CODE_SHUTDOWN, ev_info.info); @@ -537,10 +557,10 @@ int smc_ism_init(void) { int rc = 0; -#if IS_ENABLED(CONFIG_ISM) smc_ism_v2_capable = false; - memset(smc_ism_v2_system_eid, 0, SMC_MAX_EID_LEN); + smc_ism_create_system_eid(); +#if IS_ENABLED(CONFIG_ISM) rc = ism_register_client(&smc_ism_client); #endif return rc; diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 832b2f42d79f..ffff40c30a06 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -15,6 +15,9 @@ #include "smc.h" +#define SMC_VIRTUAL_ISM_CHID_MASK 0xFF00 +#define SMC_ISM_IDENT_MASK 0x00FFFF + struct smcd_dev_list { /* List of SMCD devices */ struct list_head list; struct mutex mutex; /* Protects list of devices */ @@ -28,9 +31,16 @@ struct smc_ism_vlanid { /* VLAN id set on ISM device */ refcount_t refcnt; /* Reference count */ }; +struct smc_ism_seid { + u8 seid_string[24]; + u8 serial_number[4]; + u8 type[4]; +}; + struct smcd_dev; -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *dev); +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *dev); void smc_ism_set_conn(struct smc_connection *conn); void smc_ism_unset_conn(struct smc_connection *conn); int smc_ism_get_vlan(struct smcd_dev *dev, unsigned short vlan_id); @@ -56,4 +66,22 @@ static inline int smc_ism_write(struct smcd_dev *smcd, u64 dmb_tok, return rc < 0 ? rc : 0; } +static inline bool __smc_ism_is_virtual(u16 chid) +{ + /* CHIDs in range of 0xFF00 to 0xFFFF are reserved + * for virtual ISM device. + * + * loopback-ism: 0xFFFF + * virtio-ism: 0xFF00 ~ 0xFFFE + */ + return ((chid & 0xFF00) == 0xFF00); +} + +static inline bool smc_ism_is_virtual(struct smcd_dev *smcd) +{ + u16 chid = smcd->ops->get_chid(smcd); + + return __smc_ism_is_virtual(chid); +} + #endif diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index 11775401df68..9f2c58c5a86b 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -1103,8 +1103,8 @@ static void smc_pnet_find_ism_by_pnetid(struct net_device *ndev, list_for_each_entry(ismdev, &smcd_dev_list.list, list) { if (smc_pnet_match(ismdev->pnetid, ndev_pnetid) && !ismdev->going_away && - (!ini->ism_peer_gid[0] || - !smc_ism_cantalk(ini->ism_peer_gid[0], ini->vlan_id, + (!ini->ism_peer_gid[0].gid || + !smc_ism_cantalk(&ini->ism_peer_gid[0], ini->vlan_id, ismdev))) { ini->ism_dev[0] = ismdev; break; diff --git a/net/tipc/link.c b/net/tipc/link.c index d0143823658d..0716eb5c8a31 100644 --- a/net/tipc/link.c +++ b/net/tipc/link.c @@ -82,10 +82,7 @@ struct tipc_stats { * struct tipc_link - TIPC link data structure * @addr: network address of link's peer node * @name: link name character string - * @media_addr: media address to use when sending messages over link - * @timer: link timer * @net: pointer to namespace struct - * @refcnt: reference counter for permanent references (owner node & timer) * @peer_session: link session # being used by peer end of link * @peer_bearer_id: bearer id used by link's peer endpoint * @bearer_id: local bearer id used by link @@ -94,31 +91,19 @@ struct tipc_stats { * @state: current state of link FSM * @peer_caps: bitmap describing capabilities of peer node * @silent_intv_cnt: # of timer intervals without any reception from peer - * @proto_msg: template for control messages generated by link - * @pmsg: convenience pointer to "proto_msg" field * @priority: current link priority * @net_plane: current link network plane ('A' through 'H') * @mon_state: cookie with information needed by link monitor - * @backlog_limit: backlog queue congestion thresholds (indexed by importance) - * @exp_msg_count: # of tunnelled messages expected during link changeover - * @reset_rcv_checkpt: seq # of last acknowledged message at time of link reset * @mtu: current maximum packet size for this link * @advertised_mtu: advertised own mtu when link is being established - * @transmitq: queue for sent, non-acked messages * @backlogq: queue for messages waiting to be sent - * @snt_nxt: next sequence number to use for outbound messages * @ackers: # of peers that needs to ack each packet before it can be released * @acked: # last packet acked by a certain peer. Used for broadcast. * @rcv_nxt: next sequence number to expect for inbound messages - * @deferred_queue: deferred queue saved OOS b'cast message received from node - * @unacked_window: # of inbound messages rx'd without ack'ing back to peer * @inputq: buffer queue for messages to be delivered upwards * @namedq: buffer queue for name table messages to be delivered upwards - * @next_out: ptr to first unsent outbound message in queue * @wakeupq: linked list of wakeup msgs waiting for link congestion to abate - * @long_msg_seq_no: next identifier to use for outbound fragmented messages * @reasm_buf: head of partially reassembled inbound message fragments - * @bc_rcvr: marks that this is a broadcast receiver link * @stats: collects statistics regarding link activity * @session: session to be used by link * @snd_nxt_state: next send seq number diff --git a/net/wireless/chan.c b/net/wireless/chan.c index dfb4893421d7..ceb9174c5c3d 100644 --- a/net/wireless/chan.c +++ b/net/wireless/chan.c @@ -515,9 +515,83 @@ static u32 cfg80211_get_end_freq(u32 center_freq, return end_freq; } +static bool +cfg80211_dfs_permissive_check_wdev(struct cfg80211_registered_device *rdev, + enum nl80211_iftype iftype, + struct wireless_dev *wdev, + struct ieee80211_channel *chan) +{ + unsigned int link_id; + + for_each_valid_link(wdev, link_id) { + struct ieee80211_channel *other_chan = NULL; + struct cfg80211_chan_def chandef = {}; + int ret; + + /* In order to avoid daisy chaining only allow BSS STA */ + if (wdev->iftype != NL80211_IFTYPE_STATION || + !wdev->links[link_id].client.current_bss) + continue; + + other_chan = + wdev->links[link_id].client.current_bss->pub.channel; + + if (!other_chan) + continue; + + if (chan == other_chan) + return true; + + /* continue if we can't get the channel */ + ret = rdev_get_channel(rdev, wdev, link_id, &chandef); + if (ret) + continue; + + if (cfg80211_is_sub_chan(&chandef, chan, false)) + return true; + } + + return false; +} + +/* + * Check if P2P GO is allowed to operate on a DFS channel + */ +static bool cfg80211_dfs_permissive_chan(struct wiphy *wiphy, + enum nl80211_iftype iftype, + struct ieee80211_channel *chan) +{ + struct wireless_dev *wdev; + struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy); + + lockdep_assert_held(&rdev->wiphy.mtx); + + if (!wiphy_ext_feature_isset(&rdev->wiphy, + NL80211_EXT_FEATURE_DFS_CONCURRENT) || + !(chan->flags & IEEE80211_CHAN_DFS_CONCURRENT)) + return false; + + /* only valid for P2P GO */ + if (iftype != NL80211_IFTYPE_P2P_GO) + return false; + + /* + * Allow only if there's a concurrent BSS + */ + list_for_each_entry(wdev, &rdev->wiphy.wdev_list, list) { + bool ret = cfg80211_dfs_permissive_check_wdev(rdev, iftype, + wdev, chan); + if (ret) + return ret; + } + + return false; +} + static int cfg80211_get_chans_dfs_required(struct wiphy *wiphy, u32 center_freq, - u32 bandwidth) + u32 bandwidth, + enum nl80211_iftype iftype) { struct ieee80211_channel *c; u32 freq, start_freq, end_freq; @@ -530,9 +604,11 @@ static int cfg80211_get_chans_dfs_required(struct wiphy *wiphy, if (!c) return -EINVAL; - if (c->flags & IEEE80211_CHAN_RADAR) + if (c->flags & IEEE80211_CHAN_RADAR && + !cfg80211_dfs_permissive_chan(wiphy, iftype, c)) return 1; } + return 0; } @@ -558,7 +634,7 @@ int cfg80211_chandef_dfs_required(struct wiphy *wiphy, ret = cfg80211_get_chans_dfs_required(wiphy, ieee80211_chandef_to_khz(chandef), - width); + width, iftype); if (ret < 0) return ret; else if (ret > 0) @@ -569,7 +645,7 @@ int cfg80211_chandef_dfs_required(struct wiphy *wiphy, ret = cfg80211_get_chans_dfs_required(wiphy, MHZ_TO_KHZ(chandef->center_freq2), - width); + width, iftype); if (ret < 0) return ret; else if (ret > 0) @@ -1337,15 +1413,19 @@ static bool _cfg80211_reg_can_beacon(struct wiphy *wiphy, bool check_no_ir) { bool res; - u32 prohibited_flags = IEEE80211_CHAN_DISABLED | - IEEE80211_CHAN_RADAR; + u32 prohibited_flags = IEEE80211_CHAN_DISABLED; + int dfs_required; trace_cfg80211_reg_can_beacon(wiphy, chandef, iftype, check_no_ir); if (check_no_ir) prohibited_flags |= IEEE80211_CHAN_NO_IR; - if (cfg80211_chandef_dfs_required(wiphy, chandef, iftype) > 0 && + dfs_required = cfg80211_chandef_dfs_required(wiphy, chandef, iftype); + if (dfs_required != 0) + prohibited_flags |= IEEE80211_CHAN_RADAR; + + if (dfs_required > 0 && cfg80211_chandef_dfs_available(wiphy, chandef)) { /* We can skip IEEE80211_CHAN_NO_IR if chandef dfs available */ prohibited_flags = IEEE80211_CHAN_DISABLED; diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index 8b45fb420f4c..60877b532993 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -1201,6 +1201,15 @@ static int nl80211_msg_put_channel(struct sk_buff *msg, struct wiphy *wiphy, if ((chan->flags & IEEE80211_CHAN_NO_EHT) && nla_put_flag(msg, NL80211_FREQUENCY_ATTR_NO_EHT)) goto nla_put_failure; + if ((chan->flags & IEEE80211_CHAN_DFS_CONCURRENT) && + nla_put_flag(msg, NL80211_FREQUENCY_ATTR_DFS_CONCURRENT)) + goto nla_put_failure; + if ((chan->flags & IEEE80211_CHAN_NO_UHB_VLP_CLIENT) && + nla_put_flag(msg, NL80211_FREQUENCY_ATTR_NO_UHB_VLP_CLIENT)) + goto nla_put_failure; + if ((chan->flags & IEEE80211_CHAN_NO_UHB_AFC_CLIENT) && + nla_put_flag(msg, NL80211_FREQUENCY_ATTR_NO_UHB_AFC_CLIENT)) + goto nla_put_failure; } if (nla_put_u32(msg, NL80211_FREQUENCY_ATTR_MAX_TX_POWER, @@ -19471,6 +19480,7 @@ void cfg80211_ch_switch_notify(struct net_device *dev, break; } + cfg80211_schedule_channels_check(wdev); cfg80211_sched_dfs_chan_update(rdev); nl80211_ch_switch_notify(rdev, dev, link_id, chandef, GFP_KERNEL, @@ -20228,6 +20238,20 @@ nla_put_failure: } EXPORT_SYMBOL(cfg80211_update_owe_info_event); +void cfg80211_schedule_channels_check(struct wireless_dev *wdev) +{ + struct wiphy *wiphy = wdev->wiphy; + + /* Schedule channels check if NO_IR or DFS relaxations are supported */ + if (wdev->iftype == NL80211_IFTYPE_STATION && + (wiphy_ext_feature_isset(wiphy, + NL80211_EXT_FEATURE_DFS_CONCURRENT) || + (IS_ENABLED(CONFIG_CFG80211_REG_RELAX_NO_IR) && + wiphy->regulatory_flags & REGULATORY_ENABLE_RELAX_NO_IR))) + reg_check_channels(); +} +EXPORT_SYMBOL(cfg80211_schedule_channels_check); + /* initialisation/exit functions */ int __init nl80211_init(void) diff --git a/net/wireless/reg.c b/net/wireless/reg.c index 2ef4f6cc7a32..2741b626919a 100644 --- a/net/wireless/reg.c +++ b/net/wireless/reg.c @@ -1593,6 +1593,12 @@ static u32 map_regdom_flags(u32 rd_flags) channel_flags |= IEEE80211_CHAN_NO_320MHZ; if (rd_flags & NL80211_RRF_NO_EHT) channel_flags |= IEEE80211_CHAN_NO_EHT; + if (rd_flags & NL80211_RRF_DFS_CONCURRENT) + channel_flags |= IEEE80211_CHAN_DFS_CONCURRENT; + if (rd_flags & NL80211_RRF_NO_UHB_VLP_CLIENT) + channel_flags |= IEEE80211_CHAN_NO_UHB_VLP_CLIENT; + if (rd_flags & NL80211_RRF_NO_UHB_AFC_CLIENT) + channel_flags |= IEEE80211_CHAN_NO_UHB_AFC_CLIENT; if (rd_flags & NL80211_RRF_PSD) channel_flags |= IEEE80211_CHAN_PSD; return channel_flags; @@ -2478,7 +2484,7 @@ static void reg_check_chans_work(struct work_struct *work) rtnl_unlock(); } -static void reg_check_channels(void) +void reg_check_channels(void) { /* * Give usermode a chance to do something nicer (move to another diff --git a/net/wireless/reg.h b/net/wireless/reg.h index a703e53c23ee..a02ef5609f52 100644 --- a/net/wireless/reg.h +++ b/net/wireless/reg.h @@ -181,6 +181,11 @@ bool reg_dfs_domain_same(struct wiphy *wiphy1, struct wiphy *wiphy2); */ int reg_reload_regdb(void); +/** + * reg_check_channels - schedule regulatory enforcement + */ +void reg_check_channels(void); + extern const u8 shipped_regdb_certs[]; extern unsigned int shipped_regdb_certs_len; extern const u8 extra_regdb_certs[]; diff --git a/net/wireless/scan.c b/net/wireless/scan.c index 3d260c99c348..cf2131671eb6 100644 --- a/net/wireless/scan.c +++ b/net/wireless/scan.c @@ -1818,15 +1818,15 @@ __cfg80211_bss_update(struct cfg80211_registered_device *rdev, bool signal_valid, unsigned long ts) { struct cfg80211_internal_bss *found = NULL; + struct cfg80211_bss_ies *ies; if (WARN_ON(!tmp->pub.channel)) - return NULL; + goto free_ies; tmp->ts = ts; - if (WARN_ON(!rcu_access_pointer(tmp->pub.ies))) { - return NULL; - } + if (WARN_ON(!rcu_access_pointer(tmp->pub.ies))) + goto free_ies; found = rb_find_bss(rdev, tmp, BSS_CMP_REGULAR); @@ -1836,7 +1836,6 @@ __cfg80211_bss_update(struct cfg80211_registered_device *rdev, } else { struct cfg80211_internal_bss *new; struct cfg80211_internal_bss *hidden; - struct cfg80211_bss_ies *ies; /* * create a copy -- the "res" variable that is passed in @@ -1845,15 +1844,8 @@ __cfg80211_bss_update(struct cfg80211_registered_device *rdev, */ new = kzalloc(sizeof(*new) + rdev->wiphy.bss_priv_size, GFP_ATOMIC); - if (!new) { - ies = (void *)rcu_dereference(tmp->pub.beacon_ies); - if (ies) - kfree_rcu(ies, rcu_head); - ies = (void *)rcu_dereference(tmp->pub.proberesp_ies); - if (ies) - kfree_rcu(ies, rcu_head); - return NULL; - } + if (!new) + goto free_ies; memcpy(new, tmp, sizeof(*new)); new->refcount = 1; INIT_LIST_HEAD(&new->hidden_list); @@ -1871,8 +1863,12 @@ __cfg80211_bss_update(struct cfg80211_registered_device *rdev, list_add(&new->hidden_list, &hidden->hidden_list); hidden->refcount++; + + ies = (void *)rcu_dereference(new->pub.beacon_ies); rcu_assign_pointer(new->pub.beacon_ies, hidden->pub.beacon_ies); + if (ies) + kfree_rcu(ies, rcu_head); } } else { /* @@ -1909,6 +1905,16 @@ __cfg80211_bss_update(struct cfg80211_registered_device *rdev, bss_ref_get(rdev, found); return found; + +free_ies: + ies = (void *)rcu_dereference(tmp->pub.beacon_ies); + if (ies) + kfree_rcu(ies, rcu_head); + ies = (void *)rcu_dereference(tmp->pub.proberesp_ies); + if (ies) + kfree_rcu(ies, rcu_head); + + return NULL; } struct cfg80211_internal_bss * @@ -2848,6 +2854,36 @@ cfg80211_inform_bss_data(struct wiphy *wiphy, } EXPORT_SYMBOL(cfg80211_inform_bss_data); +static bool cfg80211_uhb_power_type_valid(const u8 *ie, + size_t ielen, + const u32 flags) +{ + const struct element *tmp; + struct ieee80211_he_operation *he_oper; + + tmp = cfg80211_find_ext_elem(WLAN_EID_EXT_HE_OPERATION, ie, ielen); + if (tmp && tmp->datalen >= sizeof(*he_oper) + 1) { + const struct ieee80211_he_6ghz_oper *he_6ghz_oper; + + he_oper = (void *)&tmp->data[1]; + he_6ghz_oper = ieee80211_he_6ghz_oper(he_oper); + + if (!he_6ghz_oper) + return false; + + switch (u8_get_bits(he_6ghz_oper->control, + IEEE80211_HE_6GHZ_OPER_CTRL_REG_INFO)) { + case IEEE80211_6GHZ_CTRL_REG_LPI_AP: + return true; + case IEEE80211_6GHZ_CTRL_REG_SP_AP: + return !(flags & IEEE80211_CHAN_NO_UHB_AFC_CLIENT); + case IEEE80211_6GHZ_CTRL_REG_VLP_AP: + return !(flags & IEEE80211_CHAN_NO_UHB_VLP_CLIENT); + } + } + return false; +} + /* cfg80211_inform_bss_width_frame helper */ static struct cfg80211_bss * cfg80211_inform_single_bss_frame_data(struct wiphy *wiphy, @@ -2906,6 +2942,14 @@ cfg80211_inform_single_bss_frame_data(struct wiphy *wiphy, if (!channel) return NULL; + if (channel->band == NL80211_BAND_6GHZ && + !cfg80211_uhb_power_type_valid(variable, ielen, channel->flags)) { + data->restrict_use = 1; + data->use_for = 0; + data->cannot_use_reasons = + NL80211_BSS_CANNOT_USE_UHB_PWR_MISMATCH; + } + if (ext) { const struct ieee80211_s1g_bcn_compat_ie *compat; const struct element *elem; @@ -3150,10 +3194,9 @@ void cfg80211_update_assoc_bss_entry(struct wireless_dev *wdev, if (new) { /* to save time, update IEs for transmitting bss only */ - if (cfg80211_update_known_bss(rdev, cbss, new, false)) { - new->pub.proberesp_ies = NULL; - new->pub.beacon_ies = NULL; - } + cfg80211_update_known_bss(rdev, cbss, new, false); + new->pub.proberesp_ies = NULL; + new->pub.beacon_ies = NULL; list_for_each_entry_safe(nontrans_bss, tmp, &new->pub.nontrans_list, diff --git a/net/wireless/sme.c b/net/wireless/sme.c index acfe66da7109..195c8532734b 100644 --- a/net/wireless/sme.c +++ b/net/wireless/sme.c @@ -1394,6 +1394,8 @@ void __cfg80211_disconnected(struct net_device *dev, const u8 *ie, #endif schedule_work(&cfg80211_disconnect_work); + + cfg80211_schedule_channels_check(wdev); } void cfg80211_disconnected(struct net_device *dev, u16 reason, diff --git a/net/x25/af_x25.c b/net/x25/af_x25.c index aad8ffeaee04..f7a7c7798c3b 100644 --- a/net/x25/af_x25.c +++ b/net/x25/af_x25.c @@ -704,7 +704,7 @@ static int x25_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len) rc = -EINVAL; } release_sock(sk); - SOCK_DEBUG(sk, "x25_bind: socket is bound\n"); + net_dbg_ratelimited("x25_bind: socket is bound\n"); out: return rc; } @@ -1165,10 +1165,10 @@ static int x25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) goto out; } - SOCK_DEBUG(sk, "x25_sendmsg: sendto: Addresses built.\n"); + net_dbg_ratelimited("x25_sendmsg: sendto: Addresses built.\n"); /* Build a packet */ - SOCK_DEBUG(sk, "x25_sendmsg: sendto: building packet.\n"); + net_dbg_ratelimited("x25_sendmsg: sendto: building packet.\n"); if ((msg->msg_flags & MSG_OOB) && len > 32) len = 32; @@ -1187,7 +1187,7 @@ static int x25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) /* * Put the data on the end */ - SOCK_DEBUG(sk, "x25_sendmsg: Copying user data\n"); + net_dbg_ratelimited("x25_sendmsg: Copying user data\n"); skb_reset_transport_header(skb); skb_put(skb, len); @@ -1211,7 +1211,7 @@ static int x25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) /* * Push down the X.25 header */ - SOCK_DEBUG(sk, "x25_sendmsg: Building X.25 Header.\n"); + net_dbg_ratelimited("x25_sendmsg: Building X.25 Header.\n"); if (msg->msg_flags & MSG_OOB) { if (x25->neighbour->extended) { @@ -1245,8 +1245,8 @@ static int x25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) skb->data[0] |= X25_Q_BIT; } - SOCK_DEBUG(sk, "x25_sendmsg: Built header.\n"); - SOCK_DEBUG(sk, "x25_sendmsg: Transmitting buffer\n"); + net_dbg_ratelimited("x25_sendmsg: Built header.\n"); + net_dbg_ratelimited("x25_sendmsg: Transmitting buffer\n"); rc = -ENOTCONN; if (sk->sk_state != TCP_ESTABLISHED) diff --git a/net/x25/x25_facilities.c b/net/x25/x25_facilities.c index 8e1a49b0c0dc..6dadb217e101 100644 --- a/net/x25/x25_facilities.c +++ b/net/x25/x25_facilities.c @@ -282,7 +282,7 @@ int x25_negotiate_facilities(struct sk_buff *skb, struct sock *sk, * They want reverse charging, we won't accept it. */ if ((theirs.reverse & 0x01 ) && (ours->reverse & 0x01)) { - SOCK_DEBUG(sk, "X.25: rejecting reverse charging request\n"); + net_dbg_ratelimited("X.25: rejecting reverse charging request\n"); return -1; } @@ -294,11 +294,11 @@ int x25_negotiate_facilities(struct sk_buff *skb, struct sock *sk, int ours_in = ours->throughput & 0x0f; int ours_out = ours->throughput & 0xf0; if (!ours_in || theirs_in < ours_in) { - SOCK_DEBUG(sk, "X.25: inbound throughput negotiated\n"); + net_dbg_ratelimited("X.25: inbound throughput negotiated\n"); new->throughput = (new->throughput & 0xf0) | theirs_in; } if (!ours_out || theirs_out < ours_out) { - SOCK_DEBUG(sk, + net_dbg_ratelimited( "X.25: outbound throughput negotiated\n"); new->throughput = (new->throughput & 0x0f) | theirs_out; } @@ -306,22 +306,22 @@ int x25_negotiate_facilities(struct sk_buff *skb, struct sock *sk, if (theirs.pacsize_in && theirs.pacsize_out) { if (theirs.pacsize_in < ours->pacsize_in) { - SOCK_DEBUG(sk, "X.25: packet size inwards negotiated down\n"); + net_dbg_ratelimited("X.25: packet size inwards negotiated down\n"); new->pacsize_in = theirs.pacsize_in; } if (theirs.pacsize_out < ours->pacsize_out) { - SOCK_DEBUG(sk, "X.25: packet size outwards negotiated down\n"); + net_dbg_ratelimited("X.25: packet size outwards negotiated down\n"); new->pacsize_out = theirs.pacsize_out; } } if (theirs.winsize_in && theirs.winsize_out) { if (theirs.winsize_in < ours->winsize_in) { - SOCK_DEBUG(sk, "X.25: window size inwards negotiated down\n"); + net_dbg_ratelimited("X.25: window size inwards negotiated down\n"); new->winsize_in = theirs.winsize_in; } if (theirs.winsize_out < ours->winsize_out) { - SOCK_DEBUG(sk, "X.25: window size outwards negotiated down\n"); + net_dbg_ratelimited("X.25: window size outwards negotiated down\n"); new->winsize_out = theirs.winsize_out; } } diff --git a/net/x25/x25_out.c b/net/x25/x25_out.c index dbc0940bf35f..f8922b0e23a4 100644 --- a/net/x25/x25_out.c +++ b/net/x25/x25_out.c @@ -72,7 +72,7 @@ int x25_output(struct sock *sk, struct sk_buff *skb) kfree_skb(skb); return sent; } - SOCK_DEBUG(sk, "x25_output: fragment alloc" + net_dbg_ratelimited("x25_output: fragment alloc" " failed, err=%d, %d bytes " "sent\n", err, sent); return err; diff --git a/samples/bpf/cpustat_user.c b/samples/bpf/cpustat_user.c index ab90bb08a2b4..356f756cba0d 100644 --- a/samples/bpf/cpustat_user.c +++ b/samples/bpf/cpustat_user.c @@ -66,10 +66,10 @@ static void cpu_stat_print(void) printf("CPU-%-6d ", j); for (i = 0; i < MAX_CSTATE_ENTRIES; i++) - printf("%-11ld ", data->cstate[i] / 1000000); + printf("%-11lu ", data->cstate[i] / 1000000); for (i = 0; i < MAX_PSTATE_ENTRIES; i++) - printf("%-11ld ", data->pstate[i] / 1000000); + printf("%-11lu ", data->pstate[i] / 1000000); printf("\n"); } diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 77ceea575dc7..2324cc42b017 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -188,6 +188,9 @@ enum libbpf_tristate { !!sym; \ }) +#define __arg_ctx __attribute__((btf_decl_tag("arg:ctx"))) +#define __arg_nonnull __attribute((btf_decl_tag("arg:nonnull"))) + #ifndef ___bpf_concat #define ___bpf_concat(a, b) a ## b #endif diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index ac54ebc0629f..ebcfb2147fbd 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -4355,6 +4355,8 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Dat scn = elf_sec_by_idx(obj, sec_idx); scn_data = elf_sec_data(obj, scn); + if (!scn_data) + return -LIBBPF_ERRNO__FORMAT; relo_sec_name = elf_sec_str(obj, shdr->sh_name); sec_name = elf_sec_name(obj, scn); diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c index 52a2901e8bd0..16bca56002ab 100644 --- a/tools/lib/bpf/linker.c +++ b/tools/lib/bpf/linker.c @@ -719,6 +719,9 @@ static int linker_sanity_check_elf(struct src_obj *obj) return -EINVAL; } + if (is_dwarf_sec_name(sec->sec_name)) + continue; + if (sec->shdr->sh_addralign && !is_pow_of_2(sec->shdr->sh_addralign)) { pr_warn("ELF section #%zu alignment %llu is non pow-of-2 alignment in %s\n", sec->sec_idx, (long long unsigned)sec->shdr->sh_addralign, diff --git a/tools/testing/selftests/bpf/benchs/bench_htab_mem.c b/tools/testing/selftests/bpf/benchs/bench_htab_mem.c index 9146d3f414d2..926ee822143e 100644 --- a/tools/testing/selftests/bpf/benchs/bench_htab_mem.c +++ b/tools/testing/selftests/bpf/benchs/bench_htab_mem.c @@ -335,6 +335,7 @@ static void htab_mem_report_final(struct bench_res res[], int res_cnt) " peak memory usage %7.2lfMiB\n", loop_mean, loop_stddev, mem_mean, mem_stddev, peak_mem / 1048576.0); + close(ctx.fd); cleanup_cgroup_environment(); } diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c index 8ec73fdfcdab..f29fc789c14b 100644 --- a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +++ b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c @@ -348,7 +348,8 @@ static void test_func_sockmap_update(void) } static void test_obj_load_failure_common(const char *obj_file, - const char *target_obj_file) + const char *target_obj_file, + const char *exp_msg) { /* * standalone test that asserts failure to load freplace prog @@ -356,6 +357,7 @@ static void test_obj_load_failure_common(const char *obj_file, */ struct bpf_object *obj = NULL, *pkt_obj; struct bpf_program *prog; + char log_buf[64 * 1024]; int err, pkt_fd; __u32 duration = 0; @@ -374,11 +376,21 @@ static void test_obj_load_failure_common(const char *obj_file, err = bpf_program__set_attach_target(prog, pkt_fd, NULL); ASSERT_OK(err, "set_attach_target"); + log_buf[0] = '\0'; + if (exp_msg) + bpf_program__set_log_buf(prog, log_buf, sizeof(log_buf)); + if (env.verbosity > VERBOSE_NONE) + bpf_program__set_log_level(prog, 2); + /* It should fail to load the program */ err = bpf_object__load(obj); + if (env.verbosity > VERBOSE_NONE && exp_msg) /* we overtook log */ + printf("VERIFIER LOG:\n================\n%s\n================\n", log_buf); if (CHECK(!err, "bpf_obj_load should fail", "err %d\n", err)) goto close_prog; + if (exp_msg) + ASSERT_HAS_SUBSTR(log_buf, exp_msg, "fail_msg"); close_prog: bpf_object__close(obj); bpf_object__close(pkt_obj); @@ -388,14 +400,24 @@ static void test_func_replace_return_code(void) { /* test invalid return code in the replaced program */ test_obj_load_failure_common("./freplace_connect_v4_prog.bpf.o", - "./connect4_prog.bpf.o"); + "./connect4_prog.bpf.o", NULL); } static void test_func_map_prog_compatibility(void) { /* test with spin lock map value in the replaced program */ test_obj_load_failure_common("./freplace_attach_probe.bpf.o", - "./test_attach_probe.bpf.o"); + "./test_attach_probe.bpf.o", NULL); +} + +static void test_func_replace_unreliable(void) +{ + /* freplace'ing unreliable main prog should fail with error + * "Cannot replace static functions" + */ + test_obj_load_failure_common("freplace_unreliable_prog.bpf.o", + "./verifier_btf_unreliable_prog.bpf.o", + "Cannot replace static functions"); } static void test_func_replace_global_func(void) @@ -563,6 +585,8 @@ void serial_test_fexit_bpf2bpf(void) test_func_replace_return_code(); if (test__start_subtest("func_map_prog_compatibility")) test_func_map_prog_compatibility(); + if (test__start_subtest("func_replace_unreliable")) + test_func_replace_unreliable(); if (test__start_subtest("func_replace_multi")) test_func_replace_multi(); if (test__start_subtest("fmod_ret_freplace")) diff --git a/tools/testing/selftests/bpf/prog_tests/log_fixup.c b/tools/testing/selftests/bpf/prog_tests/log_fixup.c index effd78b2a657..7a3fa2ff567b 100644 --- a/tools/testing/selftests/bpf/prog_tests/log_fixup.c +++ b/tools/testing/selftests/bpf/prog_tests/log_fixup.c @@ -169,9 +169,9 @@ void test_log_fixup(void) if (test__start_subtest("bad_core_relo_trunc_none")) bad_core_relo(0, TRUNC_NONE /* full buf */); if (test__start_subtest("bad_core_relo_trunc_partial")) - bad_core_relo(300, TRUNC_PARTIAL /* truncate original log a bit */); + bad_core_relo(280, TRUNC_PARTIAL /* truncate original log a bit */); if (test__start_subtest("bad_core_relo_trunc_full")) - bad_core_relo(210, TRUNC_FULL /* truncate also libbpf's message patch */); + bad_core_relo(220, TRUNC_FULL /* truncate also libbpf's message patch */); if (test__start_subtest("bad_core_relo_subprog")) bad_core_relo_subprog(); if (test__start_subtest("missing_map")) diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c index 0c9abd279e18..820d0bcfc474 100644 --- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c +++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c @@ -590,12 +590,7 @@ static void range_cond(enum num_t t, struct range x, struct range y, *newy = range(t, max_t(t, x.a, y.a), min_t(t, x.b, y.b)); break; case OP_NE: - /* generic case, can't derive more information */ - *newx = range(t, x.a, x.b); - *newy = range(t, y.a, y.b); - break; - - /* below extended logic is not supported by verifier just yet */ + /* below logic is supported by the verifier now */ if (x.a == x.b && x.a == y.a) { /* X is a constant matching left side of Y */ *newx = range(t, x.a, x.b); @@ -2097,10 +2092,22 @@ static struct subtest_case crafted_cases[] = { {U32, S32, {0, U32_MAX}, {U32_MAX, U32_MAX}}, - {S32, U64, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}, {(u32)(s32)-255, 0}}, - {S32, S64, {(u32)(s32)S32_MIN, (u32)(s32)-255}, {(u32)(s32)-2, 0}}, - {S32, S64, {0, 1}, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}}, - {S32, U32, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}}, + {S32, U64, {(u32)S32_MIN, (u32)S32_MIN}, {(u32)(s32)-255, 0}}, + {S32, S64, {(u32)S32_MIN, (u32)(s32)-255}, {(u32)(s32)-2, 0}}, + {S32, S64, {0, 1}, {(u32)S32_MIN, (u32)S32_MIN}}, + {S32, U32, {(u32)S32_MIN, (u32)S32_MIN}, {(u32)S32_MIN, (u32)S32_MIN}}, + + /* edge overlap testings for BPF_NE */ + {U64, U64, {0, U64_MAX}, {U64_MAX, U64_MAX}}, + {U64, U64, {0, U64_MAX}, {0, 0}}, + {S64, U64, {S64_MIN, 0}, {S64_MIN, S64_MIN}}, + {S64, U64, {S64_MIN, 0}, {0, 0}}, + {S64, U64, {S64_MIN, S64_MAX}, {S64_MAX, S64_MAX}}, + {U32, U32, {0, U32_MAX}, {0, 0}}, + {U32, U32, {0, U32_MAX}, {U32_MAX, U32_MAX}}, + {S32, U32, {(u32)S32_MIN, 0}, {0, 0}}, + {S32, U32, {(u32)S32_MIN, 0}, {(u32)S32_MIN, (u32)S32_MIN}}, + {S32, U32, {(u32)S32_MIN, S32_MAX}, {S32_MAX, S32_MAX}}, }; /* Go over crafted hard-coded cases. This is fast, so we do it as part of diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c index ac49ec25211d..d62c5bf00e71 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -14,6 +14,7 @@ #include "verifier_bpf_get_stack.skel.h" #include "verifier_bswap.skel.h" #include "verifier_btf_ctx_access.skel.h" +#include "verifier_btf_unreliable_prog.skel.h" #include "verifier_cfg.skel.h" #include "verifier_cgroup_inv_retcode.skel.h" #include "verifier_cgroup_skb.skel.h" @@ -125,6 +126,7 @@ void test_verifier_bounds_mix_sign_unsign(void) { RUN(verifier_bounds_mix_sign_u void test_verifier_bpf_get_stack(void) { RUN(verifier_bpf_get_stack); } void test_verifier_bswap(void) { RUN(verifier_bswap); } void test_verifier_btf_ctx_access(void) { RUN(verifier_btf_ctx_access); } +void test_verifier_btf_unreliable_prog(void) { RUN(verifier_btf_unreliable_prog); } void test_verifier_cfg(void) { RUN(verifier_cfg); } void test_verifier_cgroup_inv_retcode(void) { RUN(verifier_cgroup_inv_retcode); } void test_verifier_cgroup_skb(void) { RUN(verifier_cgroup_skb); } diff --git a/tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c b/tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c index 0fa564a5cc5b..9fe9c4a4e8f6 100644 --- a/tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c +++ b/tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c @@ -78,7 +78,7 @@ int BPF_PROG(cgrp_kfunc_acquire_fp, struct cgroup *cgrp, const char *path) } SEC("kretprobe/cgroup_destroy_locked") -__failure __msg("reg type unsupported for arg#0 function") +__failure __msg("calling kernel function bpf_cgroup_acquire is not allowed") int BPF_PROG(cgrp_kfunc_acquire_unsafe_kretprobe, struct cgroup *cgrp) { struct cgroup *acquired; diff --git a/tools/testing/selftests/bpf/progs/freplace_unreliable_prog.c b/tools/testing/selftests/bpf/progs/freplace_unreliable_prog.c new file mode 100644 index 000000000000..624078abf3de --- /dev/null +++ b/tools/testing/selftests/bpf/progs/freplace_unreliable_prog.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2020 Facebook + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +SEC("freplace/btf_unreliable_kprobe") +/* context type is what BPF verifier expects for kprobe context, but target + * program has `stuct whatever *ctx` argument, so freplace operation will be + * rejected with the following message: + * + * arg0 replace_btf_unreliable_kprobe(struct pt_regs *) doesn't match btf_unreliable_kprobe(struct whatever *) + */ +int replace_btf_unreliable_kprobe(bpf_user_pt_regs_t *ctx) +{ + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/task_kfunc_failure.c b/tools/testing/selftests/bpf/progs/task_kfunc_failure.c index dcdea3127086..ad88a3796ddf 100644 --- a/tools/testing/selftests/bpf/progs/task_kfunc_failure.c +++ b/tools/testing/selftests/bpf/progs/task_kfunc_failure.c @@ -248,7 +248,7 @@ int BPF_PROG(task_kfunc_from_pid_no_null_check, struct task_struct *task, u64 cl } SEC("lsm/task_free") -__failure __msg("reg type unsupported for arg#0 function") +__failure __msg("R1 must be a rcu pointer") int BPF_PROG(task_kfunc_from_lsm_task_free, struct task_struct *task) { struct task_struct *acquired; diff --git a/tools/testing/selftests/bpf/progs/test_bpf_ma.c b/tools/testing/selftests/bpf/progs/test_bpf_ma.c index b685a4aba6bd..069db9085e78 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_ma.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_ma.c @@ -17,7 +17,7 @@ struct generic_map_value { char _license[] SEC("license") = "GPL"; -const unsigned int data_sizes[] = {8, 16, 32, 64, 96, 128, 192, 256, 512, 1024, 2048, 4096}; +const unsigned int data_sizes[] = {16, 32, 64, 96, 128, 192, 256, 512, 1024, 2048, 4096}; const volatile unsigned int data_btf_ids[ARRAY_SIZE(data_sizes)] = {}; int err = 0; @@ -166,7 +166,7 @@ static __always_inline void batch_percpu_free(struct bpf_map *map, unsigned int batch_percpu_free((struct bpf_map *)(&array_percpu_##size), batch, idx); \ } while (0) -DEFINE_ARRAY_WITH_KPTR(8); +/* kptr doesn't support bin_data_8 which is a zero-sized array */ DEFINE_ARRAY_WITH_KPTR(16); DEFINE_ARRAY_WITH_KPTR(32); DEFINE_ARRAY_WITH_KPTR(64); @@ -198,21 +198,20 @@ int test_batch_alloc_free(void *ctx) if ((u32)bpf_get_current_pid_tgid() != pid) return 0; - /* Alloc 128 8-bytes objects in batch to trigger refilling, - * then free 128 8-bytes objects in batch to trigger freeing. + /* Alloc 128 16-bytes objects in batch to trigger refilling, + * then free 128 16-bytes objects in batch to trigger freeing. */ - CALL_BATCH_ALLOC_FREE(8, 128, 0); - CALL_BATCH_ALLOC_FREE(16, 128, 1); - CALL_BATCH_ALLOC_FREE(32, 128, 2); - CALL_BATCH_ALLOC_FREE(64, 128, 3); - CALL_BATCH_ALLOC_FREE(96, 128, 4); - CALL_BATCH_ALLOC_FREE(128, 128, 5); - CALL_BATCH_ALLOC_FREE(192, 128, 6); - CALL_BATCH_ALLOC_FREE(256, 128, 7); - CALL_BATCH_ALLOC_FREE(512, 64, 8); - CALL_BATCH_ALLOC_FREE(1024, 32, 9); - CALL_BATCH_ALLOC_FREE(2048, 16, 10); - CALL_BATCH_ALLOC_FREE(4096, 8, 11); + CALL_BATCH_ALLOC_FREE(16, 128, 0); + CALL_BATCH_ALLOC_FREE(32, 128, 1); + CALL_BATCH_ALLOC_FREE(64, 128, 2); + CALL_BATCH_ALLOC_FREE(96, 128, 3); + CALL_BATCH_ALLOC_FREE(128, 128, 4); + CALL_BATCH_ALLOC_FREE(192, 128, 5); + CALL_BATCH_ALLOC_FREE(256, 128, 6); + CALL_BATCH_ALLOC_FREE(512, 64, 7); + CALL_BATCH_ALLOC_FREE(1024, 32, 8); + CALL_BATCH_ALLOC_FREE(2048, 16, 9); + CALL_BATCH_ALLOC_FREE(4096, 8, 10); return 0; } @@ -223,21 +222,20 @@ int test_free_through_map_free(void *ctx) if ((u32)bpf_get_current_pid_tgid() != pid) return 0; - /* Alloc 128 8-bytes objects in batch to trigger refilling, + /* Alloc 128 16-bytes objects in batch to trigger refilling, * then free these objects through map free. */ - CALL_BATCH_ALLOC(8, 128, 0); - CALL_BATCH_ALLOC(16, 128, 1); - CALL_BATCH_ALLOC(32, 128, 2); - CALL_BATCH_ALLOC(64, 128, 3); - CALL_BATCH_ALLOC(96, 128, 4); - CALL_BATCH_ALLOC(128, 128, 5); - CALL_BATCH_ALLOC(192, 128, 6); - CALL_BATCH_ALLOC(256, 128, 7); - CALL_BATCH_ALLOC(512, 64, 8); - CALL_BATCH_ALLOC(1024, 32, 9); - CALL_BATCH_ALLOC(2048, 16, 10); - CALL_BATCH_ALLOC(4096, 8, 11); + CALL_BATCH_ALLOC(16, 128, 0); + CALL_BATCH_ALLOC(32, 128, 1); + CALL_BATCH_ALLOC(64, 128, 2); + CALL_BATCH_ALLOC(96, 128, 3); + CALL_BATCH_ALLOC(128, 128, 4); + CALL_BATCH_ALLOC(192, 128, 5); + CALL_BATCH_ALLOC(256, 128, 6); + CALL_BATCH_ALLOC(512, 64, 7); + CALL_BATCH_ALLOC(1024, 32, 8); + CALL_BATCH_ALLOC(2048, 16, 9); + CALL_BATCH_ALLOC(4096, 8, 10); return 0; } @@ -251,17 +249,17 @@ int test_batch_percpu_alloc_free(void *ctx) /* Alloc 128 16-bytes per-cpu objects in batch to trigger refilling, * then free 128 16-bytes per-cpu objects in batch to trigger freeing. */ - CALL_BATCH_PERCPU_ALLOC_FREE(16, 128, 1); - CALL_BATCH_PERCPU_ALLOC_FREE(32, 128, 2); - CALL_BATCH_PERCPU_ALLOC_FREE(64, 128, 3); - CALL_BATCH_PERCPU_ALLOC_FREE(96, 128, 4); - CALL_BATCH_PERCPU_ALLOC_FREE(128, 128, 5); - CALL_BATCH_PERCPU_ALLOC_FREE(192, 128, 6); - CALL_BATCH_PERCPU_ALLOC_FREE(256, 128, 7); - CALL_BATCH_PERCPU_ALLOC_FREE(512, 64, 8); - CALL_BATCH_PERCPU_ALLOC_FREE(1024, 32, 9); - CALL_BATCH_PERCPU_ALLOC_FREE(2048, 16, 10); - CALL_BATCH_PERCPU_ALLOC_FREE(4096, 8, 11); + CALL_BATCH_PERCPU_ALLOC_FREE(16, 128, 0); + CALL_BATCH_PERCPU_ALLOC_FREE(32, 128, 1); + CALL_BATCH_PERCPU_ALLOC_FREE(64, 128, 2); + CALL_BATCH_PERCPU_ALLOC_FREE(96, 128, 3); + CALL_BATCH_PERCPU_ALLOC_FREE(128, 128, 4); + CALL_BATCH_PERCPU_ALLOC_FREE(192, 128, 5); + CALL_BATCH_PERCPU_ALLOC_FREE(256, 128, 6); + CALL_BATCH_PERCPU_ALLOC_FREE(512, 64, 7); + CALL_BATCH_PERCPU_ALLOC_FREE(1024, 32, 8); + CALL_BATCH_PERCPU_ALLOC_FREE(2048, 16, 9); + CALL_BATCH_PERCPU_ALLOC_FREE(4096, 8, 10); return 0; } @@ -275,17 +273,17 @@ int test_percpu_free_through_map_free(void *ctx) /* Alloc 128 16-bytes per-cpu objects in batch to trigger refilling, * then free these object through map free. */ - CALL_BATCH_PERCPU_ALLOC(16, 128, 1); - CALL_BATCH_PERCPU_ALLOC(32, 128, 2); - CALL_BATCH_PERCPU_ALLOC(64, 128, 3); - CALL_BATCH_PERCPU_ALLOC(96, 128, 4); - CALL_BATCH_PERCPU_ALLOC(128, 128, 5); - CALL_BATCH_PERCPU_ALLOC(192, 128, 6); - CALL_BATCH_PERCPU_ALLOC(256, 128, 7); - CALL_BATCH_PERCPU_ALLOC(512, 64, 8); - CALL_BATCH_PERCPU_ALLOC(1024, 32, 9); - CALL_BATCH_PERCPU_ALLOC(2048, 16, 10); - CALL_BATCH_PERCPU_ALLOC(4096, 8, 11); + CALL_BATCH_PERCPU_ALLOC(16, 128, 0); + CALL_BATCH_PERCPU_ALLOC(32, 128, 1); + CALL_BATCH_PERCPU_ALLOC(64, 128, 2); + CALL_BATCH_PERCPU_ALLOC(96, 128, 3); + CALL_BATCH_PERCPU_ALLOC(128, 128, 4); + CALL_BATCH_PERCPU_ALLOC(192, 128, 5); + CALL_BATCH_PERCPU_ALLOC(256, 128, 6); + CALL_BATCH_PERCPU_ALLOC(512, 64, 7); + CALL_BATCH_PERCPU_ALLOC(1024, 32, 8); + CALL_BATCH_PERCPU_ALLOC(2048, 16, 9); + CALL_BATCH_PERCPU_ALLOC(4096, 8, 10); return 0; } diff --git a/tools/testing/selftests/bpf/progs/test_global_func5.c b/tools/testing/selftests/bpf/progs/test_global_func5.c index cc55aedaf82d..257c0569ff98 100644 --- a/tools/testing/selftests/bpf/progs/test_global_func5.c +++ b/tools/testing/selftests/bpf/progs/test_global_func5.c @@ -26,7 +26,7 @@ int f3(int val, struct __sk_buff *skb) } SEC("tc") -__failure __msg("expected pointer to ctx, but got PTR") +__failure __msg("expects pointer to ctx") int global_func5(struct __sk_buff *skb) { return f1(skb) + f2(2, skb) + f3(3, skb); diff --git a/tools/testing/selftests/bpf/progs/verifier_bounds.c b/tools/testing/selftests/bpf/progs/verifier_bounds.c index ec430b71730b..960998f16306 100644 --- a/tools/testing/selftests/bpf/progs/verifier_bounds.c +++ b/tools/testing/selftests/bpf/progs/verifier_bounds.c @@ -1075,4 +1075,66 @@ l0_%=: r0 = 0; \ : __clobber_all); } +SEC("tc") +__description("bounds check with JMP_NE for reg edge") +__success __retval(0) +__naked void reg_not_equal_const(void) +{ + asm volatile (" \ + r6 = r1; \ + r1 = 0; \ + *(u64*)(r10 - 8) = r1; \ + call %[bpf_get_prandom_u32]; \ + r4 = r0; \ + r4 &= 7; \ + if r4 != 0 goto l0_%=; \ + r0 = 0; \ + exit; \ +l0_%=: r1 = r6; \ + r2 = 0; \ + r3 = r10; \ + r3 += -8; \ + r5 = 0; \ + /* The 4th argument of bpf_skb_store_bytes is defined as \ + * ARG_CONST_SIZE, so 0 is not allowed. The 'r4 != 0' \ + * is providing us this exclusion of zero from initial \ + * [0, 7] range. \ + */ \ + call %[bpf_skb_store_bytes]; \ + r0 = 0; \ + exit; \ +" : + : __imm(bpf_get_prandom_u32), + __imm(bpf_skb_store_bytes) + : __clobber_all); +} + +SEC("tc") +__description("bounds check with JMP_EQ for reg edge") +__success __retval(0) +__naked void reg_equal_const(void) +{ + asm volatile (" \ + r6 = r1; \ + r1 = 0; \ + *(u64*)(r10 - 8) = r1; \ + call %[bpf_get_prandom_u32]; \ + r4 = r0; \ + r4 &= 7; \ + if r4 == 0 goto l0_%=; \ + r1 = r6; \ + r2 = 0; \ + r3 = r10; \ + r3 += -8; \ + r5 = 0; \ + /* Just the same as what we do in reg_not_equal_const() */ \ + call %[bpf_skb_store_bytes]; \ +l0_%=: r0 = 0; \ + exit; \ +" : + : __imm(bpf_get_prandom_u32), + __imm(bpf_skb_store_bytes) + : __clobber_all); +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/verifier_btf_unreliable_prog.c b/tools/testing/selftests/bpf/progs/verifier_btf_unreliable_prog.c new file mode 100644 index 000000000000..36e033a2e02c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_btf_unreliable_prog.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2017 Facebook + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_core_read.h> +#include "bpf_misc.h" + +struct whatever {}; + +SEC("kprobe") +__success __log_level(2) +/* context type is wrong, making it impossible to freplace this program */ +int btf_unreliable_kprobe(struct whatever *ctx) +{ + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c b/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c index bd696a431244..9eeb2d89cda8 100644 --- a/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c +++ b/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c @@ -1,12 +1,11 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ -#include <stdbool.h> -#include <errno.h> -#include <string.h> -#include <linux/bpf.h> +#include <vmlinux.h> #include <bpf/bpf_helpers.h> #include "bpf_misc.h" +#include "xdp_metadata.h" +#include "bpf_kfuncs.h" int arr[1]; int unkn_idx; @@ -98,4 +97,96 @@ int unguarded_unsupp_global_called(void) return global_unsupp(&x); } +long stack[128]; + +__weak int subprog_nullable_ptr_bad(int *p) +{ + return (*p) * 2; /* bad, missing null check */ +} + +SEC("?raw_tp") +__failure __log_level(2) +__msg("invalid mem access 'mem_or_null'") +int arg_tag_nullable_ptr_fail(void *ctx) +{ + int x = 42; + + return subprog_nullable_ptr_bad(&x); +} + +__noinline __weak int subprog_nonnull_ptr_good(int *p1 __arg_nonnull, int *p2 __arg_nonnull) +{ + return (*p1) * (*p2); /* good, no need for NULL checks */ +} + +int x = 47; + +SEC("?raw_tp") +__success __log_level(2) +int arg_tag_nonnull_ptr_good(void *ctx) +{ + int y = 74; + + return subprog_nonnull_ptr_good(&x, &y); +} + +/* this global subprog can be now called from many types of entry progs, each + * with different context type + */ +__weak int subprog_ctx_tag(void *ctx __arg_ctx) +{ + return bpf_get_stack(ctx, stack, sizeof(stack), 0); +} + +SEC("?raw_tp") +__success __log_level(2) +int arg_tag_ctx_raw_tp(void *ctx) +{ + return subprog_ctx_tag(ctx); +} + +SEC("?tp") +__success __log_level(2) +int arg_tag_ctx_tp(void *ctx) +{ + return subprog_ctx_tag(ctx); +} + +SEC("?kprobe") +__success __log_level(2) +int arg_tag_ctx_kprobe(void *ctx) +{ + return subprog_ctx_tag(ctx); +} + +__weak int subprog_dynptr(struct bpf_dynptr *dptr) +{ + long *d, t, buf[1] = {}; + + d = bpf_dynptr_data(dptr, 0, sizeof(long)); + if (!d) + return 0; + + t = *d + 1; + + d = bpf_dynptr_slice(dptr, 0, &buf, sizeof(long)); + if (!d) + return t; + + t = *d + 2; + + return t; +} + +SEC("?xdp") +__success __log_level(2) +int arg_tag_dynptr(struct xdp_md *ctx) +{ + struct bpf_dynptr dptr; + + bpf_dynptr_from_xdp(ctx, 0, &dptr); + + return subprog_dynptr(&dptr); +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index cd2fb43eea61..74954f6a8f94 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -6,6 +6,7 @@ export skip_rc=4 export timeout_rc=124 export logfile=/dev/stdout export per_test_logging= +export RUN_IN_NETNS= # Defaults for "settings" file fields: # "timeout" how many seconds to let each test run before running @@ -47,7 +48,7 @@ run_one() { DIR="$1" TEST="$2" - NUM="$3" + local test_num="$3" BASENAME_TEST=$(basename $TEST) @@ -141,6 +142,33 @@ run_one() fi } +in_netns() +{ + local name=$1 + ip netns exec $name bash <<-EOF + BASE_DIR=$BASE_DIR + source $BASE_DIR/kselftest/runner.sh + logfile=$logfile + run_one $DIR $TEST $test_num + EOF +} + +run_in_netns() +{ + local netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX) + local tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)" + ip netns add $netns + if [ $? -ne 0 ]; then + echo "# Warning: Create namespace failed for $BASENAME_TEST" + echo "not ok $test_num selftests: $DIR: $BASENAME_TEST # Create NS failed" + fi + ip -n $netns link set lo up + in_netns $netns &> $tmplog + ip netns del $netns &> /dev/null + cat $tmplog + rm -f $tmplog +} + run_many() { echo "TAP version 13" @@ -155,6 +183,12 @@ run_many() logfile="/tmp/$BASENAME_TEST" cat /dev/null > "$logfile" fi - run_one "$DIR" "$TEST" "$test_num" + if [ -n "$RUN_IN_NETNS" ]; then + run_in_netns & + else + run_one "$DIR" "$TEST" "$test_num" + fi done + + wait } diff --git a/tools/testing/selftests/net/forwarding/ethtool_mm.sh b/tools/testing/selftests/net/forwarding/ethtool_mm.sh index 39e736f30322..50d5bfb17ef1 100755 --- a/tools/testing/selftests/net/forwarding/ethtool_mm.sh +++ b/tools/testing/selftests/net/forwarding/ethtool_mm.sh @@ -25,6 +25,10 @@ traffic_test() local after= local delta= + if [ ${has_pmac_stats[$if]} = false ]; then + src="aggregate" + fi + before=$(ethtool_std_stats_get $if "eth-mac" "FramesTransmittedOK" $src) $MZ $if -q -c $num_pkts -p 64 -b bcast -t ip -R $PREEMPTIBLE_PRIO @@ -155,15 +159,48 @@ manual_failed_verification_h2_to_h1() manual_failed_verification $h2 $h1 } +smallest_supported_add_frag_size() +{ + local iface=$1 + local rx_min_frag_size= + + rx_min_frag_size=$(ethtool --json --show-mm $iface | \ + jq '.[]."rx-min-frag-size"') + + if [ $rx_min_frag_size -le 60 ]; then + echo 0 + elif [ $rx_min_frag_size -le 124 ]; then + echo 1 + elif [ $rx_min_frag_size -le 188 ]; then + echo 2 + elif [ $rx_min_frag_size -le 252 ]; then + echo 3 + else + echo "$iface: RX min frag size $rx_min_frag_size cannot be advertised over LLDP" + exit 1 + fi +} + +expected_add_frag_size() +{ + local iface=$1 + local requested=$2 + local min=$(smallest_supported_add_frag_size $iface) + + [ $requested -le $min ] && echo $min || echo $requested +} + lldp_change_add_frag_size() { local add_frag_size=$1 + local pattern= lldptool -T -i $h1 -V addEthCaps addFragSize=$add_frag_size >/dev/null # Wait for TLVs to be received sleep 2 - lldptool -i $h2 -t -n -V addEthCaps | \ - grep -q "Additional fragment size: $add_frag_size" + pattern=$(printf "Additional fragment size: %d" \ + $(expected_add_frag_size $h1 $add_frag_size)) + lldptool -i $h2 -t -n -V addEthCaps | grep -q "$pattern" } lldp() @@ -284,6 +321,13 @@ for netif in ${NETIFS[@]}; do echo "SKIP: $netif does not support MAC Merge" exit $ksft_skip fi + + if check_ethtool_pmac_std_stats_support $netif eth-mac; then + has_pmac_stats[$netif]=true + else + has_pmac_stats[$netif]=false + echo "$netif does not report pMAC statistics, falling back to aggregate" + fi done trap cleanup EXIT diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh index e3740163c384..69ef2a40df21 100755 --- a/tools/testing/selftests/net/forwarding/lib.sh +++ b/tools/testing/selftests/net/forwarding/lib.sh @@ -155,6 +155,15 @@ check_ethtool_counter_group_support() fi } +check_ethtool_pmac_std_stats_support() +{ + local dev=$1; shift + local grp=$1; shift + + [ 0 -ne $(ethtool --json -S $dev --all-groups --src pmac 2>/dev/null \ + | jq ".[].\"$grp\" | length") ] +} + check_locked_port_support() { if ! bridge -d link show | grep -q " locked"; then diff --git a/tools/testing/selftests/net/gre_gso.sh b/tools/testing/selftests/net/gre_gso.sh index 3224651db97b..5100d90f92d2 100755 --- a/tools/testing/selftests/net/gre_gso.sh +++ b/tools/testing/selftests/net/gre_gso.sh @@ -2,10 +2,8 @@ # SPDX-License-Identifier: GPL-2.0 # This test is for checking GRE GSO. - +source lib.sh ret=0 -# Kselftest framework requirement - SKIP code is 4. -ksft_skip=4 # all tests in this script. Can be overridden with -t option TESTS="gre_gso" @@ -13,8 +11,6 @@ TESTS="gre_gso" VERBOSE=0 PAUSE_ON_FAIL=no PAUSE=no -IP="ip -netns ns1" -NS_EXEC="ip netns exec ns1" TMPFILE=`mktemp` PID= @@ -50,13 +46,13 @@ log_test() setup() { set -e - ip netns add ns1 - ip netns set ns1 auto - $IP link set dev lo up + setup_ns ns1 + IP="ip -netns $ns1" + NS_EXEC="ip netns exec $ns1" ip link add veth0 type veth peer name veth1 ip link set veth0 up - ip link set veth1 netns ns1 + ip link set veth1 netns $ns1 $IP link set veth1 name veth0 $IP link set veth0 up @@ -70,7 +66,7 @@ cleanup() [ -n "$PID" ] && kill $PID ip link del dev gre1 &> /dev/null ip link del dev veth0 &> /dev/null - ip netns del ns1 + cleanup_ns $ns1 } get_linklocal() @@ -145,7 +141,7 @@ gre6_gso_test() setup a1=$(get_linklocal veth0) - a2=$(get_linklocal veth0 ns1) + a2=$(get_linklocal veth0 $ns1) gre_create_tun $a1 $a2 diff --git a/tools/testing/selftests/net/gro.sh b/tools/testing/selftests/net/gro.sh index 342ad27f631b..19352f106c1d 100755 --- a/tools/testing/selftests/net/gro.sh +++ b/tools/testing/selftests/net/gro.sh @@ -23,11 +23,11 @@ run_test() { # on every try. for tries in {1..3}; do # Actual test starts here - ip netns exec server_ns ./gro "${ARGS[@]}" "--rx" "--iface" "server" \ + ip netns exec $server_ns ./gro "${ARGS[@]}" "--rx" "--iface" "server" \ 1>>log.txt & server_pid=$! sleep 0.5 # to allow for socket init - ip netns exec client_ns ./gro "${ARGS[@]}" "--iface" "client" \ + ip netns exec $client_ns ./gro "${ARGS[@]}" "--iface" "client" \ 1>>log.txt wait "${server_pid}" exit_code=$? diff --git a/tools/testing/selftests/net/ip_local_port_range.c b/tools/testing/selftests/net/ip_local_port_range.c index 75e3fdacdf73..0f217a1cc837 100644 --- a/tools/testing/selftests/net/ip_local_port_range.c +++ b/tools/testing/selftests/net/ip_local_port_range.c @@ -146,6 +146,12 @@ FIXTURE_VARIANT_ADD(ip_local_port_range, ip4_stcp) { .so_protocol = IPPROTO_SCTP, }; +FIXTURE_VARIANT_ADD(ip_local_port_range, ip4_mptcp) { + .so_domain = AF_INET, + .so_type = SOCK_STREAM, + .so_protocol = IPPROTO_MPTCP, +}; + FIXTURE_VARIANT_ADD(ip_local_port_range, ip6_tcp) { .so_domain = AF_INET6, .so_type = SOCK_STREAM, @@ -164,6 +170,12 @@ FIXTURE_VARIANT_ADD(ip_local_port_range, ip6_stcp) { .so_protocol = IPPROTO_SCTP, }; +FIXTURE_VARIANT_ADD(ip_local_port_range, ip6_mptcp) { + .so_domain = AF_INET6, + .so_type = SOCK_STREAM, + .so_protocol = IPPROTO_MPTCP, +}; + TEST_F(ip_local_port_range, invalid_option_value) { __u16 val16; diff --git a/tools/testing/selftests/net/mptcp/diag.sh b/tools/testing/selftests/net/mptcp/diag.sh index 95b498efacd1..04fcb8a077c9 100755 --- a/tools/testing/selftests/net/mptcp/diag.sh +++ b/tools/testing/selftests/net/mptcp/diag.sh @@ -56,7 +56,7 @@ __chk_nr() local command="$1" local expected=$2 local msg="$3" - local skip="${4:-SKIP}" + local skip="${4-SKIP}" local nr nr=$(eval $command) @@ -182,6 +182,15 @@ chk_msk_inuse() __chk_nr get_msk_inuse $expected "$msg" 0 } +# $1: cestab nr +chk_msk_cestab() +{ + local cestab=$1 + + __chk_nr "mptcp_lib_get_counter ${ns} MPTcpExtMPCurrEstab" \ + "${cestab}" "....chk ${cestab} cestab" "" +} + wait_connected() { local listener_ns="${1}" @@ -219,9 +228,11 @@ chk_msk_nr 2 "after MPC handshake " chk_msk_remote_key_nr 2 "....chk remote_key" chk_msk_fallback_nr 0 "....chk no fallback" chk_msk_inuse 2 "....chk 2 msk in use" +chk_msk_cestab 2 flush_pids chk_msk_inuse 0 "....chk 0 msk in use after flush" +chk_msk_cestab 0 echo "a" | \ timeout ${timeout_test} \ @@ -237,9 +248,11 @@ echo "b" | \ wait_connected $ns 10001 chk_msk_fallback_nr 1 "check fallback" chk_msk_inuse 1 "....chk 1 msk in use" +chk_msk_cestab 1 flush_pids chk_msk_inuse 0 "....chk 0 msk in use after flush" +chk_msk_cestab 0 NR_CLIENTS=100 for I in `seq 1 $NR_CLIENTS`; do @@ -261,9 +274,11 @@ done wait_msk_nr $((NR_CLIENTS*2)) "many msk socket present" chk_msk_inuse $((NR_CLIENTS*2)) "....chk many msk in use" +chk_msk_cestab $((NR_CLIENTS*2)) flush_pids chk_msk_inuse 0 "....chk 0 msk in use after flush" +chk_msk_cestab 0 mptcp_lib_result_print_all_tap exit $ret diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index 87590a43b50d..3a5b63026191 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -56,6 +56,8 @@ unset FAILING_LINKS unset test_linkfail unset addr_nr_ns1 unset addr_nr_ns2 +unset cestab_ns1 +unset cestab_ns2 unset sflags unset fastclose unset fullmesh @@ -976,6 +978,34 @@ pm_nl_set_endpoint() fi } +chk_cestab_nr() +{ + local ns=$1 + local cestab=$2 + local count + + print_check "cestab $cestab" + count=$(mptcp_lib_get_counter ${ns} "MPTcpExtMPCurrEstab") + if [ -z "$count" ]; then + print_skip + elif [ "$count" != "$cestab" ]; then + fail_test "got $count current establish[s] expected $cestab" + else + print_ok + fi +} + +# $1 namespace 1, $2 namespace 2 +check_cestab() +{ + if [ -n "${cestab_ns1}" ]; then + chk_cestab_nr ${1} ${cestab_ns1} + fi + if [ -n "${cestab_ns2}" ]; then + chk_cestab_nr ${2} ${cestab_ns2} + fi +} + do_transfer() { local listener_ns="$1" @@ -1089,6 +1119,7 @@ do_transfer() local cpid=$! pm_nl_set_endpoint $listener_ns $connector_ns $connect_addr + check_cestab $listener_ns $connector_ns wait $cpid local retc=$? @@ -2477,47 +2508,52 @@ add_tests() if reset "add single subflow"; then pm_nl_set_limits $ns1 0 1 pm_nl_set_limits $ns2 0 1 - addr_nr_ns2=1 speed=slow \ + addr_nr_ns2=1 speed=slow cestab_ns2=1 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 1 1 1 + chk_cestab_nr $ns2 0 fi # add signal address if reset "add signal address"; then pm_nl_set_limits $ns1 0 1 pm_nl_set_limits $ns2 1 1 - addr_nr_ns1=1 speed=slow \ + addr_nr_ns1=1 speed=slow cestab_ns1=1 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 1 1 1 chk_add_nr 1 1 + chk_cestab_nr $ns1 0 fi # add multiple subflows if reset "add multiple subflows"; then pm_nl_set_limits $ns1 0 2 pm_nl_set_limits $ns2 0 2 - addr_nr_ns2=2 speed=slow \ + addr_nr_ns2=2 speed=slow cestab_ns2=1 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 2 2 2 + chk_cestab_nr $ns2 0 fi # add multiple subflows IPv6 if reset "add multiple subflows IPv6"; then pm_nl_set_limits $ns1 0 2 pm_nl_set_limits $ns2 0 2 - addr_nr_ns2=2 speed=slow \ + addr_nr_ns2=2 speed=slow cestab_ns2=1 \ run_tests $ns1 $ns2 dead:beef:1::1 chk_join_nr 2 2 2 + chk_cestab_nr $ns2 0 fi # add multiple addresses IPv6 if reset "add multiple addresses IPv6"; then pm_nl_set_limits $ns1 0 2 pm_nl_set_limits $ns2 2 2 - addr_nr_ns1=2 speed=slow \ + addr_nr_ns1=2 speed=slow cestab_ns1=1 \ run_tests $ns1 $ns2 dead:beef:1::1 chk_join_nr 2 2 2 chk_add_nr 2 2 + chk_cestab_nr $ns1 0 fi } diff --git a/tools/testing/selftests/net/netns-name.sh b/tools/testing/selftests/net/netns-name.sh index 7d3d3fc99461..6974474c26f3 100755 --- a/tools/testing/selftests/net/netns-name.sh +++ b/tools/testing/selftests/net/netns-name.sh @@ -1,9 +1,9 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 +source lib.sh set -o pipefail -NS=netns-name-test DEV=dummy-dev0 DEV2=dummy-dev1 ALT_NAME=some-alt-name @@ -11,7 +11,7 @@ ALT_NAME=some-alt-name RET_CODE=0 cleanup() { - ip netns del $NS + cleanup_ns $NS $test_ns } trap cleanup EXIT @@ -21,50 +21,50 @@ fail() { RET_CODE=1 } -ip netns add $NS +setup_ns NS test_ns # # Test basic move without a rename # ip -netns $NS link add name $DEV type dummy || fail -ip -netns $NS link set dev $DEV netns 1 || +ip -netns $NS link set dev $DEV netns $test_ns || fail "Can't perform a netns move" -ip link show dev $DEV >> /dev/null || fail "Device not found after move" -ip link del $DEV || fail +ip -netns $test_ns link show dev $DEV >> /dev/null || fail "Device not found after move" +ip -netns $test_ns link del $DEV || fail # # Test move with a conflict # -ip link add name $DEV type dummy +ip -netns $test_ns link add name $DEV type dummy ip -netns $NS link add name $DEV type dummy || fail -ip -netns $NS link set dev $DEV netns 1 2> /dev/null && +ip -netns $NS link set dev $DEV netns $test_ns 2> /dev/null && fail "Performed a netns move with a name conflict" -ip link show dev $DEV >> /dev/null || fail "Device not found after move" +ip -netns $test_ns link show dev $DEV >> /dev/null || fail "Device not found after move" ip -netns $NS link del $DEV || fail -ip link del $DEV || fail +ip -netns $test_ns link del $DEV || fail # # Test move with a conflict and rename # -ip link add name $DEV type dummy +ip -netns $test_ns link add name $DEV type dummy ip -netns $NS link add name $DEV type dummy || fail -ip -netns $NS link set dev $DEV netns 1 name $DEV2 || +ip -netns $NS link set dev $DEV netns $test_ns name $DEV2 || fail "Can't perform a netns move with rename" -ip link del $DEV2 || fail -ip link del $DEV || fail +ip -netns $test_ns link del $DEV2 || fail +ip -netns $test_ns link del $DEV || fail # # Test dup alt-name with netns move # -ip link add name $DEV type dummy || fail -ip link property add dev $DEV altname $ALT_NAME || fail +ip -netns $test_ns link add name $DEV type dummy || fail +ip -netns $test_ns link property add dev $DEV altname $ALT_NAME || fail ip -netns $NS link add name $DEV2 type dummy || fail ip -netns $NS link property add dev $DEV2 altname $ALT_NAME || fail -ip -netns $NS link set dev $DEV2 netns 1 2> /dev/null && +ip -netns $NS link set dev $DEV2 netns $test_ns 2> /dev/null && fail "Moved with alt-name dup" -ip link del $DEV || fail +ip -netns $test_ns link del $DEV || fail ip -netns $NS link del $DEV2 || fail # @@ -72,11 +72,11 @@ ip -netns $NS link del $DEV2 || fail # ip -netns $NS link add name $DEV type dummy || fail ip -netns $NS link property add dev $DEV altname $ALT_NAME || fail -ip -netns $NS link set dev $DEV netns 1 || fail -ip link show dev $ALT_NAME >> /dev/null || fail "Can't find alt-name after move" -ip -netns $NS link show dev $ALT_NAME 2> /dev/null && +ip -netns $NS link set dev $DEV netns $test_ns || fail +ip -netns $test_ns link show dev $ALT_NAME >> /dev/null || fail "Can't find alt-name after move" +ip -netns $NS link show dev $ALT_NAME 2> /dev/null && fail "Can still find alt-name after move" -ip link del $DEV || fail +ip -netns $test_ns link del $DEV || fail echo -ne "$(basename $0) \t\t\t\t" if [ $RET_CODE -eq 0 ]; then diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh index b3b2dc5a630c..175d3d1d773b 100755 --- a/tools/testing/selftests/net/pmtu.sh +++ b/tools/testing/selftests/net/pmtu.sh @@ -198,8 +198,7 @@ # - pmtu_ipv6_route_change # Same as above but with IPv6 -# Kselftest framework requirement - SKIP code is 4. -ksft_skip=4 +source ./lib.sh PAUSE_ON_FAIL=no VERBOSE=0 @@ -268,16 +267,6 @@ tests=" pmtu_ipv4_route_change ipv4: PMTU exception w/route replace 1 pmtu_ipv6_route_change ipv6: PMTU exception w/route replace 1" -NS_A="ns-A" -NS_B="ns-B" -NS_C="ns-C" -NS_R1="ns-R1" -NS_R2="ns-R2" -ns_a="ip netns exec ${NS_A}" -ns_b="ip netns exec ${NS_B}" -ns_c="ip netns exec ${NS_C}" -ns_r1="ip netns exec ${NS_R1}" -ns_r2="ip netns exec ${NS_R2}" # Addressing and routing for tests with routers: four network segments, with # index SEGMENT between 1 and 4, a common prefix (PREFIX4 or PREFIX6) and an # identifier ID, which is 1 for hosts (A and B), 2 for routers (R1 and R2). @@ -543,13 +532,17 @@ setup_ip6ip6() { } setup_namespaces() { + setup_ns NS_A NS_B NS_C NS_R1 NS_R2 for n in ${NS_A} ${NS_B} ${NS_C} ${NS_R1} ${NS_R2}; do - ip netns add ${n} || return 1 - # Disable DAD, so that we don't have to wait to use the # configured IPv6 addresses ip netns exec ${n} sysctl -q net/ipv6/conf/default/accept_dad=0 done + ns_a="ip netns exec ${NS_A}" + ns_b="ip netns exec ${NS_B}" + ns_c="ip netns exec ${NS_C}" + ns_r1="ip netns exec ${NS_R1}" + ns_r2="ip netns exec ${NS_R2}" } setup_veth() { @@ -839,7 +832,7 @@ setup_bridge() { run_cmd ${ns_a} ip link set br0 up run_cmd ${ns_c} ip link add veth_C-A type veth peer name veth_A-C - run_cmd ${ns_c} ip link set veth_A-C netns ns-A + run_cmd ${ns_c} ip link set veth_A-C netns ${NS_A} run_cmd ${ns_a} ip link set veth_A-C up run_cmd ${ns_c} ip link set veth_C-A up @@ -944,9 +937,7 @@ cleanup() { done socat_pids= - for n in ${NS_A} ${NS_B} ${NS_C} ${NS_R1} ${NS_R2}; do - ip netns del ${n} 2> /dev/null - done + cleanup_all_ns ip link del veth_A-C 2>/dev/null ip link del veth_A-R1 2>/dev/null diff --git a/tools/testing/selftests/net/rtnetlink.sh b/tools/testing/selftests/net/rtnetlink.sh index 26827ea4e3e5..a10a32952f21 100755 --- a/tools/testing/selftests/net/rtnetlink.sh +++ b/tools/testing/selftests/net/rtnetlink.sh @@ -35,8 +35,7 @@ VERBOSE=0 PAUSE=no PAUSE_ON_FAIL=no -# Kselftest framework requirement - SKIP code is 4. -ksft_skip=4 +source lib.sh # set global exit status, but never reset nonzero one. check_err() @@ -517,9 +516,8 @@ kci_test_encap_fou() # test various encap methods, use netns to avoid unwanted interference kci_test_encap() { - testns="testns" local ret=0 - run_cmd ip netns add "$testns" + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP encap tests: cannot add net namespace $testns" return $ksft_skip @@ -574,6 +572,10 @@ kci_test_macsec_offload() return $ksft_skip fi + if ! mount | grep -q debugfs; then + mount -t debugfs none /sys/kernel/debug/ &> /dev/null + fi + # setup netdevsim since dummydev doesn't have offload support if [ ! -w /sys/bus/netdevsim/new_device ] ; then run_cmd modprobe -q netdevsim @@ -738,6 +740,10 @@ kci_test_ipsec_offload() sysfsnet=/sys/bus/netdevsim/devices/netdevsim0/net/ probed=false + if ! mount | grep -q debugfs; then + mount -t debugfs none /sys/kernel/debug/ &> /dev/null + fi + # setup netdevsim since dummydev doesn't have offload support if [ ! -w /sys/bus/netdevsim/new_device ] ; then run_cmd modprobe -q netdevsim @@ -836,11 +842,10 @@ EOF kci_test_gretap() { - testns="testns" DEV_NS=gretap00 local ret=0 - run_cmd ip netns add "$testns" + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP gretap tests: cannot add net namespace $testns" return $ksft_skip @@ -878,11 +883,10 @@ kci_test_gretap() kci_test_ip6gretap() { - testns="testns" DEV_NS=ip6gretap00 local ret=0 - run_cmd ip netns add "$testns" + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP ip6gretap tests: cannot add net namespace $testns" return $ksft_skip @@ -920,7 +924,6 @@ kci_test_ip6gretap() kci_test_erspan() { - testns="testns" DEV_NS=erspan00 local ret=0 run_cmd_grep "^Usage:" ip link help erspan @@ -928,7 +931,7 @@ kci_test_erspan() end_test "SKIP: erspan: iproute2 too old" return $ksft_skip fi - run_cmd ip netns add "$testns" + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP erspan tests: cannot add net namespace $testns" return $ksft_skip @@ -970,7 +973,6 @@ kci_test_erspan() kci_test_ip6erspan() { - testns="testns" DEV_NS=ip6erspan00 local ret=0 run_cmd_grep "^Usage:" ip link help ip6erspan @@ -978,7 +980,7 @@ kci_test_ip6erspan() end_test "SKIP: ip6erspan: iproute2 too old" return $ksft_skip fi - run_cmd ip netns add "$testns" + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP ip6erspan tests: cannot add net namespace $testns" return $ksft_skip @@ -1022,8 +1024,6 @@ kci_test_ip6erspan() kci_test_fdb_get() { - IP="ip -netns testns" - BRIDGE="bridge -netns testns" brdev="test-br0" vxlandev="vxlan10" test_mac=de:ad:be:ef:13:37 @@ -1037,11 +1037,13 @@ kci_test_fdb_get() return $ksft_skip fi - run_cmd ip netns add testns + setup_ns testns if [ $? -ne 0 ]; then end_test "SKIP fdb get tests: cannot add net namespace $testns" return $ksft_skip fi + IP="ip -netns $testns" + BRIDGE="bridge -netns $testns" run_cmd $IP link add "$vxlandev" type vxlan id 10 local $localip \ dstport 4789 run_cmd $IP link add name "$brdev" type bridge @@ -1052,7 +1054,7 @@ kci_test_fdb_get() run_cmd_grep "dev $vxlandev master $brdev" $BRIDGE fdb get $test_mac br "$brdev" run_cmd_grep "dev $vxlandev dst $dstip" $BRIDGE fdb get $test_mac dev "$vxlandev" self - ip netns del testns &>/dev/null + ip netns del $testns &>/dev/null if [ $ret -ne 0 ]; then end_test "FAIL: bridge fdb get" diff --git a/tools/testing/selftests/net/setup_loopback.sh b/tools/testing/selftests/net/setup_loopback.sh index e57bbfbc5208..2070b57849de 100755 --- a/tools/testing/selftests/net/setup_loopback.sh +++ b/tools/testing/selftests/net/setup_loopback.sh @@ -5,6 +5,8 @@ readonly FLUSH_PATH="/sys/class/net/${dev}/gro_flush_timeout" readonly IRQ_PATH="/sys/class/net/${dev}/napi_defer_hard_irqs" readonly FLUSH_TIMEOUT="$(< ${FLUSH_PATH})" readonly HARD_IRQS="$(< ${IRQ_PATH})" +readonly server_ns=$(mktemp -u server-XXXXXXXX) +readonly client_ns=$(mktemp -u client-XXXXXXXX) netdev_check_for_carrier() { local -r dev="$1" @@ -97,12 +99,12 @@ setup_interrupt() { setup_ns() { # Set up server_ns namespace and client_ns namespace - setup_macvlan_ns "${dev}" server_ns server "${SERVER_MAC}" - setup_macvlan_ns "${dev}" client_ns client "${CLIENT_MAC}" + setup_macvlan_ns "${dev}" ${server_ns} server "${SERVER_MAC}" + setup_macvlan_ns "${dev}" ${client_ns} client "${CLIENT_MAC}" } cleanup_ns() { - cleanup_macvlan_ns server_ns server client_ns client + cleanup_macvlan_ns ${server_ns} server ${client_ns} client } setup() { diff --git a/tools/testing/selftests/net/setup_veth.sh b/tools/testing/selftests/net/setup_veth.sh index 1003ddf7b3b2..a9a1759e035c 100644 --- a/tools/testing/selftests/net/setup_veth.sh +++ b/tools/testing/selftests/net/setup_veth.sh @@ -1,6 +1,9 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 +readonly server_ns=$(mktemp -u server-XXXXXXXX) +readonly client_ns=$(mktemp -u client-XXXXXXXX) + setup_veth_ns() { local -r link_dev="$1" local -r ns_name="$2" @@ -19,14 +22,14 @@ setup_ns() { # Set up server_ns namespace and client_ns namespace ip link add name server type veth peer name client - setup_veth_ns "${dev}" server_ns server "${SERVER_MAC}" - setup_veth_ns "${dev}" client_ns client "${CLIENT_MAC}" + setup_veth_ns "${dev}" ${server_ns} server "${SERVER_MAC}" + setup_veth_ns "${dev}" ${client_ns} client "${CLIENT_MAC}" } cleanup_ns() { local ns_name - for ns_name in client_ns server_ns; do + for ns_name in ${client_ns} ${server_ns}; do [[ -e /var/run/netns/"${ns_name}" ]] && ip netns del "${ns_name}" done } diff --git a/tools/testing/selftests/net/stress_reuseport_listen.sh b/tools/testing/selftests/net/stress_reuseport_listen.sh index 4de11da4092b..94d5d1a1c90f 100755 --- a/tools/testing/selftests/net/stress_reuseport_listen.sh +++ b/tools/testing/selftests/net/stress_reuseport_listen.sh @@ -2,18 +2,18 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2022 Meta Platforms, Inc. and affiliates. -NS='stress_reuseport_listen_ns' +source lib.sh NR_FILES=24100 SAVED_NR_FILES=$(ulimit -n) setup() { - ip netns add $NS + setup_ns NS ip netns exec $NS sysctl -q -w net.ipv6.ip_nonlocal_bind=1 ulimit -n $NR_FILES } cleanup() { - ip netns del $NS + cleanup_ns $NS ulimit -n $SAVED_NR_FILES } diff --git a/tools/testing/selftests/net/tcp_ao/Makefile b/tools/testing/selftests/net/tcp_ao/Makefile index 6343cfcf919b..8e60bae67aa9 100644 --- a/tools/testing/selftests/net/tcp_ao/Makefile +++ b/tools/testing/selftests/net/tcp_ao/Makefile @@ -17,22 +17,18 @@ TEST_IPV6_PROGS := $(TEST_BOTH_AF:%=%_ipv6) TEST_GEN_PROGS := $(TEST_IPV4_PROGS) $(TEST_IPV6_PROGS) top_srcdir := ../../../../.. -KSFT_KHDR_INSTALL := 1 include ../../lib.mk HOSTAR ?= ar -# Drop it on port to linux/master with commit 8ce72dc32578 -.DEFAULT_GOAL := all - LIBDIR := $(OUTPUT)/lib LIB := $(LIBDIR)/libaotst.a LDLIBS += $(LIB) -pthread LIBDEPS := lib/aolib.h Makefile CFLAGS := -Wall -O2 -g -D_GNU_SOURCE -fno-strict-aliasing -CFLAGS += -I ../../../../../usr/include/ -iquote $(LIBDIR) -CFLAGS += -I ../../../../include/ +CFLAGS += $(KHDR_INCLUDES) +CFLAGS += -iquote ./lib/ -I ../../../../include/ # Library LIBSRC := kconfig.c netlink.c proc.c repair.c setup.c sock.c utils.c @@ -43,6 +39,7 @@ $(LIB): $(LIBOBJ) $(HOSTAR) rcs $@ $^ $(LIBDIR)/%.o: ./lib/%.c $(LIBDEPS) + mkdir -p $(LIBDIR) $(CC) $< $(CFLAGS) $(CPPFLAGS) -o $@ -c $(TEST_GEN_PROGS): $(LIB) diff --git a/tools/testing/selftests/net/tcp_ao/bench-lookups.c b/tools/testing/selftests/net/tcp_ao/bench-lookups.c index 7be8a7d9308c..a1e6e007c291 100644 --- a/tools/testing/selftests/net/tcp_ao/bench-lookups.c +++ b/tools/testing/selftests/net/tcp_ao/bench-lookups.c @@ -46,8 +46,10 @@ static void test_add_routes(union tcp_addr *ips, size_t ips_nr) for (i = 0; i < ips_nr; i++) { union tcp_addr *p = (union tcp_addr *)&ips[i]; + int err; - if (ip_route_add(veth_name, TEST_FAMILY, this_ip_addr, *p)) + err = ip_route_add(veth_name, TEST_FAMILY, this_ip_addr, *p); + if (err && err != -EEXIST) test_error("Failed to add route"); } } diff --git a/tools/testing/selftests/net/tcp_ao/connect-deny.c b/tools/testing/selftests/net/tcp_ao/connect-deny.c index 1ca78040d8b7..185a2f6e5ff3 100644 --- a/tools/testing/selftests/net/tcp_ao/connect-deny.c +++ b/tools/testing/selftests/net/tcp_ao/connect-deny.c @@ -55,7 +55,7 @@ static void try_accept(const char *tst_name, unsigned int port, const char *pwd, err = test_wait_fd(lsk, timeout, 0); if (err == -ETIMEDOUT) { if (!fault(TIMEOUT)) - test_fail("timeouted for accept()"); + test_fail("timed out for accept()"); } else if (err < 0) { test_error("test_wait_fd()"); } else { diff --git a/tools/testing/selftests/net/tcp_ao/lib/netlink.c b/tools/testing/selftests/net/tcp_ao/lib/netlink.c index b731f2c84083..7f108493a29a 100644 --- a/tools/testing/selftests/net/tcp_ao/lib/netlink.c +++ b/tools/testing/selftests/net/tcp_ao/lib/netlink.c @@ -261,7 +261,7 @@ static int __ip_route_add(int sock, uint32_t seq, const char *intf, int family, req.nh.nlmsg_seq = seq; req.rt.rtm_family = family; req.rt.rtm_dst_len = (family == AF_INET) ? 32 : 128; - req.rt.rtm_table = RT_TABLE_MAIN; + req.rt.rtm_table = vrf; req.rt.rtm_protocol = RTPROT_BOOT; req.rt.rtm_scope = RT_SCOPE_UNIVERSE; req.rt.rtm_type = RTN_UNICAST; @@ -294,8 +294,6 @@ int ip_route_add_vrf(const char *intf, int family, ret = __ip_route_add(route_sock, route_seq++, intf, family, src, dst, vrf); - if (ret == -EEXIST) /* ignoring */ - ret = 0; close(route_sock); return ret; diff --git a/tools/testing/selftests/net/tcp_ao/lib/proc.c b/tools/testing/selftests/net/tcp_ao/lib/proc.c index 2322f4d4676d..2fb6dd8adba6 100644 --- a/tools/testing/selftests/net/tcp_ao/lib/proc.c +++ b/tools/testing/selftests/net/tcp_ao/lib/proc.c @@ -227,7 +227,7 @@ void netstat_print_diff(struct netstat *nsa, struct netstat *nsb) } if (nsb->counters_nr < nsa->counters_nr) - test_error("Unexpected: some counters dissapeared!"); + test_error("Unexpected: some counters disappeared!"); for (j = 0, i = 0; i < nsb->counters_nr; i++) { if (strcmp(nsb->counters[i].name, nsa->counters[j].name)) { @@ -244,7 +244,7 @@ void netstat_print_diff(struct netstat *nsa, struct netstat *nsb) j++; } if (j != nsa->counters_nr) - test_error("Unexpected: some counters dissapeared!"); + test_error("Unexpected: some counters disappeared!"); nsb = nsb->next; nsa = nsa->next; diff --git a/tools/testing/selftests/net/tcp_ao/lib/setup.c b/tools/testing/selftests/net/tcp_ao/lib/setup.c index 374b27c26ebd..92276f916f2f 100644 --- a/tools/testing/selftests/net/tcp_ao/lib/setup.c +++ b/tools/testing/selftests/net/tcp_ao/lib/setup.c @@ -277,22 +277,38 @@ void __test_init(unsigned int ntests, int family, unsigned int prefix, /* /proc/sys/net/core/optmem_max artifically limits the amount of memory * that can be allocated with sock_kmalloc() on each socket in the system. - * It is not virtualized, so it has to written outside test namespaces. - * To be nice a test will revert optmem back to the old value. + * It is not virtualized in v6.7, so it has to written outside test + * namespaces. To be nice a test will revert optmem back to the old value. * Keeping it simple without any file lock, which means the tests that * need to set/increase optmem value shouldn't run in parallel. * Also, not re-entrant. + * Since commit f5769faeec36 ("net: Namespace-ify sysctl_optmem_max") + * it is per-namespace, keeping logic for non-virtualized optmem_max + * for v6.7, which supports TCP-AO. */ static const char *optmem_file = "/proc/sys/net/core/optmem_max"; static size_t saved_optmem; +static int optmem_ns = -1; + +static bool is_optmem_namespaced(void) +{ + if (optmem_ns == -1) { + int old_ns = switch_save_ns(nsfd_child); + + optmem_ns = !access(optmem_file, F_OK); + switch_ns(old_ns); + } + return !!optmem_ns; +} size_t test_get_optmem(void) { + int old_ns = 0; FILE *foptmem; - int old_ns; size_t ret; - old_ns = switch_save_ns(nsfd_outside); + if (!is_optmem_namespaced()) + old_ns = switch_save_ns(nsfd_outside); foptmem = fopen(optmem_file, "r"); if (!foptmem) test_error("failed to open %s", optmem_file); @@ -300,19 +316,21 @@ size_t test_get_optmem(void) if (fscanf(foptmem, "%zu", &ret) != 1) test_error("can't read from %s", optmem_file); fclose(foptmem); - switch_ns(old_ns); + if (!is_optmem_namespaced()) + switch_ns(old_ns); return ret; } static void __test_set_optmem(size_t new, size_t *old) { + int old_ns = 0; FILE *foptmem; - int old_ns; if (old != NULL) *old = test_get_optmem(); - old_ns = switch_save_ns(nsfd_outside); + if (!is_optmem_namespaced()) + old_ns = switch_save_ns(nsfd_outside); foptmem = fopen(optmem_file, "w"); if (!foptmem) test_error("failed to open %s", optmem_file); @@ -320,7 +338,8 @@ static void __test_set_optmem(size_t new, size_t *old) if (fprintf(foptmem, "%zu", new) <= 0) test_error("can't write %zu to %s", new, optmem_file); fclose(foptmem); - switch_ns(old_ns); + if (!is_optmem_namespaced()) + switch_ns(old_ns); } static void test_revert_optmem(void) diff --git a/tools/testing/selftests/net/tcp_ao/setsockopt-closed.c b/tools/testing/selftests/net/tcp_ao/setsockopt-closed.c index 7e4601b3f6a3..452de131fa3a 100644 --- a/tools/testing/selftests/net/tcp_ao/setsockopt-closed.c +++ b/tools/testing/selftests/net/tcp_ao/setsockopt-closed.c @@ -118,7 +118,7 @@ static void setsockopt_checked(int sk, int optname, void *optval, break; default: break; - }; + } __setsockopt_checked(sk, optname, false, optval, &len, err, cmd, tst); } @@ -427,7 +427,7 @@ static void test_einval_del_key(void) sk = prepare_defs(TCP_AO_DEL_KEY, &del); del.set_current = 1; - setsockopt_checked(sk, TCP_AO_DEL_KEY, &del, ENOENT, "set non-exising current key"); + setsockopt_checked(sk, TCP_AO_DEL_KEY, &del, ENOENT, "set non-existing current key"); sk = prepare_defs(TCP_AO_DEL_KEY, &del); del.set_rnext = 1; diff --git a/tools/testing/selftests/net/tcp_ao/unsigned-md5.c b/tools/testing/selftests/net/tcp_ao/unsigned-md5.c index 7cffde02d2be..c5b568cd7d90 100644 --- a/tools/testing/selftests/net/tcp_ao/unsigned-md5.c +++ b/tools/testing/selftests/net/tcp_ao/unsigned-md5.c @@ -30,7 +30,7 @@ static void setup_vrfs(void) err = ip_route_add_vrf(veth_name, TEST_FAMILY, this_ip_addr, this_ip_dest, test_vrf_tabid); if (err) - test_error("Failed to add a route to VRF"); + test_error("Failed to add a route to VRF: %d", err); } static void try_accept(const char *tst_name, unsigned int port, @@ -72,7 +72,7 @@ static void try_accept(const char *tst_name, unsigned int port, err = test_wait_fd(lsk, timeout, 0); if (err == -ETIMEDOUT) { if (!fault(TIMEOUT)) - test_fail("timeouted for accept()"); + test_fail("timed out for accept()"); } else if (err < 0) { test_error("test_wait_fd()"); } else { @@ -494,15 +494,14 @@ out: static void client_add_ip(union tcp_addr *client, const char *ip) { - int family = TEST_FAMILY; + int err, family = TEST_FAMILY; if (inet_pton(family, ip, client) != 1) test_error("Can't convert ip address %s", ip); - if (ip_addr_add(veth_name, family, *client, TEST_PREFIX)) - test_error("Failed to add ip address"); - if (ip_route_add(veth_name, family, *client, this_ip_dest)) - test_error("Failed to add route"); + err = ip_addr_add(veth_name, family, *client, TEST_PREFIX); + if (err) + test_error("Failed to add ip address: %d", err); } static void client_add_ips(void) diff --git a/tools/testing/selftests/net/toeplitz.sh b/tools/testing/selftests/net/toeplitz.sh index da5bfd834eff..8ff172f7bb1b 100755 --- a/tools/testing/selftests/net/toeplitz.sh +++ b/tools/testing/selftests/net/toeplitz.sh @@ -147,14 +147,14 @@ setup() { setup_loopback_environment "${DEV}" # Set up server_ns namespace and client_ns namespace - setup_macvlan_ns "${DEV}" server_ns server \ + setup_macvlan_ns "${DEV}" $server_ns server \ "${SERVER_MAC}" "${SERVER_IP}" - setup_macvlan_ns "${DEV}" client_ns client \ + setup_macvlan_ns "${DEV}" $client_ns client \ "${CLIENT_MAC}" "${CLIENT_IP}" } cleanup() { - cleanup_macvlan_ns server_ns server client_ns client + cleanup_macvlan_ns $server_ns server $client_ns client cleanup_loopback "${DEV}" } @@ -170,22 +170,22 @@ if [[ "${TEST_RSS}" = true ]]; then # RPS/RFS must be disabled because they move packets between cpus, # which breaks the PACKET_FANOUT_CPU identification of RSS decisions. eval "$(get_disable_rfs_cmd) $(get_disable_rps_cmd)" \ - ip netns exec server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ + ip netns exec $server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ -d "${PORT}" -i "${DEV}" -k "${KEY}" -T 1000 \ -C "$(get_rx_irq_cpus)" -s -v & elif [[ ! -z "${RPS_MAP}" ]]; then eval "$(get_disable_rfs_cmd) $(get_set_rps_bitmaps_cmd ${RPS_MAP})" \ - ip netns exec server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ + ip netns exec $server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ -d "${PORT}" -i "${DEV}" -k "${KEY}" -T 1000 \ -r "0x${RPS_MAP}" -s -v & else - ip netns exec server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ + ip netns exec $server_ns ./toeplitz "${IP_FLAG}" "${PROTO_FLAG}" \ -d "${PORT}" -i "${DEV}" -k "${KEY}" -T 1000 -s -v & fi server_pid=$! -ip netns exec client_ns ./toeplitz_client.sh "${PROTO_FLAG}" \ +ip netns exec $client_ns ./toeplitz_client.sh "${PROTO_FLAG}" \ "${IP_FLAG}" "${SERVER_IP%%/*}" "${PORT}" & client_pid=$! diff --git a/tools/testing/selftests/net/xfrm_policy.sh b/tools/testing/selftests/net/xfrm_policy.sh index bdf450eaf60c..457789530645 100755 --- a/tools/testing/selftests/net/xfrm_policy.sh +++ b/tools/testing/selftests/net/xfrm_policy.sh @@ -18,8 +18,7 @@ # ns1: ping 10.0.2.254: does NOT pass via ipsec tunnel (exception) # ns2: ping 10.0.1.254: does NOT pass via ipsec tunnel (exception) -# Kselftest framework requirement - SKIP code is 4. -ksft_skip=4 +source lib.sh ret=0 policy_checks_ok=1 @@ -204,24 +203,24 @@ check_xfrm() { ip=$2 local lret=0 - ip netns exec ns1 ping -q -c 1 10.0.2.$ip > /dev/null + ip netns exec ${ns[1]} ping -q -c 1 10.0.2.$ip > /dev/null - check_ipt_policy_count ns3 + check_ipt_policy_count ${ns[3]} if [ $? -ne $rval ] ; then lret=1 fi - check_ipt_policy_count ns4 + check_ipt_policy_count ${ns[4]} if [ $? -ne $rval ] ; then lret=1 fi - ip netns exec ns2 ping -q -c 1 10.0.1.$ip > /dev/null + ip netns exec ${ns[2]} ping -q -c 1 10.0.1.$ip > /dev/null - check_ipt_policy_count ns3 + check_ipt_policy_count ${ns[3]} if [ $? -ne $rval ] ; then lret=1 fi - check_ipt_policy_count ns4 + check_ipt_policy_count ${ns[4]} if [ $? -ne $rval ] ; then lret=1 fi @@ -270,11 +269,11 @@ check_hthresh_repeat() i=0 for i in $(seq 1 10);do - ip -net ns1 xfrm policy update src e000:0001::0000 dst ff01::0014:0000:0001 dir in tmpl src :: dst :: proto esp mode tunnel priority 100 action allow || break - ip -net ns1 xfrm policy set hthresh6 0 28 || break + ip -net ${ns[1]} xfrm policy update src e000:0001::0000 dst ff01::0014:0000:0001 dir in tmpl src :: dst :: proto esp mode tunnel priority 100 action allow || break + ip -net ${ns[1]} xfrm policy set hthresh6 0 28 || break - ip -net ns1 xfrm policy update src e000:0001::0000 dst ff01::01 dir in tmpl src :: dst :: proto esp mode tunnel priority 100 action allow || break - ip -net ns1 xfrm policy set hthresh6 0 28 || break + ip -net ${ns[1]} xfrm policy update src e000:0001::0000 dst ff01::01 dir in tmpl src :: dst :: proto esp mode tunnel priority 100 action allow || break + ip -net ${ns[1]} xfrm policy set hthresh6 0 28 || break done if [ $i -ne 10 ] ;then @@ -347,79 +346,80 @@ if [ $? -ne 0 ];then exit $ksft_skip fi -for i in 1 2 3 4; do - ip netns add ns$i - ip -net ns$i link set lo up -done +setup_ns ns1 ns2 ns3 ns4 +ns[1]=$ns1 +ns[2]=$ns2 +ns[3]=$ns3 +ns[4]=$ns4 DEV=veth0 -ip link add $DEV netns ns1 type veth peer name eth1 netns ns3 -ip link add $DEV netns ns2 type veth peer name eth1 netns ns4 +ip link add $DEV netns ${ns[1]} type veth peer name eth1 netns ${ns[3]} +ip link add $DEV netns ${ns[2]} type veth peer name eth1 netns ${ns[4]} -ip link add $DEV netns ns3 type veth peer name veth0 netns ns4 +ip link add $DEV netns ${ns[3]} type veth peer name veth0 netns ${ns[4]} DEV=veth0 for i in 1 2; do - ip -net ns$i link set $DEV up - ip -net ns$i addr add 10.0.$i.2/24 dev $DEV - ip -net ns$i addr add dead:$i::2/64 dev $DEV - - ip -net ns$i addr add 10.0.$i.253 dev $DEV - ip -net ns$i addr add 10.0.$i.254 dev $DEV - ip -net ns$i addr add dead:$i::fd dev $DEV - ip -net ns$i addr add dead:$i::fe dev $DEV + ip -net ${ns[$i]} link set $DEV up + ip -net ${ns[$i]} addr add 10.0.$i.2/24 dev $DEV + ip -net ${ns[$i]} addr add dead:$i::2/64 dev $DEV + + ip -net ${ns[$i]} addr add 10.0.$i.253 dev $DEV + ip -net ${ns[$i]} addr add 10.0.$i.254 dev $DEV + ip -net ${ns[$i]} addr add dead:$i::fd dev $DEV + ip -net ${ns[$i]} addr add dead:$i::fe dev $DEV done for i in 3 4; do -ip -net ns$i link set eth1 up -ip -net ns$i link set veth0 up + ip -net ${ns[$i]} link set eth1 up + ip -net ${ns[$i]} link set veth0 up done -ip -net ns1 route add default via 10.0.1.1 -ip -net ns2 route add default via 10.0.2.1 +ip -net ${ns[1]} route add default via 10.0.1.1 +ip -net ${ns[2]} route add default via 10.0.2.1 -ip -net ns3 addr add 10.0.1.1/24 dev eth1 -ip -net ns3 addr add 10.0.3.1/24 dev veth0 -ip -net ns3 addr add 2001:1::1/64 dev eth1 -ip -net ns3 addr add 2001:3::1/64 dev veth0 +ip -net ${ns[3]} addr add 10.0.1.1/24 dev eth1 +ip -net ${ns[3]} addr add 10.0.3.1/24 dev veth0 +ip -net ${ns[3]} addr add 2001:1::1/64 dev eth1 +ip -net ${ns[3]} addr add 2001:3::1/64 dev veth0 -ip -net ns3 route add default via 10.0.3.10 +ip -net ${ns[3]} route add default via 10.0.3.10 -ip -net ns4 addr add 10.0.2.1/24 dev eth1 -ip -net ns4 addr add 10.0.3.10/24 dev veth0 -ip -net ns4 addr add 2001:2::1/64 dev eth1 -ip -net ns4 addr add 2001:3::10/64 dev veth0 -ip -net ns4 route add default via 10.0.3.1 +ip -net ${ns[4]} addr add 10.0.2.1/24 dev eth1 +ip -net ${ns[4]} addr add 10.0.3.10/24 dev veth0 +ip -net ${ns[4]} addr add 2001:2::1/64 dev eth1 +ip -net ${ns[4]} addr add 2001:3::10/64 dev veth0 +ip -net ${ns[4]} route add default via 10.0.3.1 for j in 4 6; do for i in 3 4;do - ip netns exec ns$i sysctl net.ipv$j.conf.eth1.forwarding=1 > /dev/null - ip netns exec ns$i sysctl net.ipv$j.conf.veth0.forwarding=1 > /dev/null + ip netns exec ${ns[$i]} sysctl net.ipv$j.conf.eth1.forwarding=1 > /dev/null + ip netns exec ${ns[$i]} sysctl net.ipv$j.conf.veth0.forwarding=1 > /dev/null done done # abuse iptables rule counter to check if ping matches a policy -ip netns exec ns3 iptables -p icmp -A FORWARD -m policy --dir out --pol ipsec -ip netns exec ns4 iptables -p icmp -A FORWARD -m policy --dir out --pol ipsec +ip netns exec ${ns[3]} iptables -p icmp -A FORWARD -m policy --dir out --pol ipsec +ip netns exec ${ns[4]} iptables -p icmp -A FORWARD -m policy --dir out --pol ipsec if [ $? -ne 0 ];then echo "SKIP: Could not insert iptables rule" - for i in 1 2 3 4;do ip netns del ns$i;done + cleanup_ns $ns1 $ns2 $ns3 $ns4 exit $ksft_skip fi # localip remoteip localnet remotenet -do_esp ns3 10.0.3.1 10.0.3.10 10.0.1.0/24 10.0.2.0/24 $SPI1 $SPI2 -do_esp ns3 dead:3::1 dead:3::10 dead:1::/64 dead:2::/64 $SPI1 $SPI2 -do_esp ns4 10.0.3.10 10.0.3.1 10.0.2.0/24 10.0.1.0/24 $SPI2 $SPI1 -do_esp ns4 dead:3::10 dead:3::1 dead:2::/64 dead:1::/64 $SPI2 $SPI1 +do_esp ${ns[3]} 10.0.3.1 10.0.3.10 10.0.1.0/24 10.0.2.0/24 $SPI1 $SPI2 +do_esp ${ns[3]} dead:3::1 dead:3::10 dead:1::/64 dead:2::/64 $SPI1 $SPI2 +do_esp ${ns[4]} 10.0.3.10 10.0.3.1 10.0.2.0/24 10.0.1.0/24 $SPI2 $SPI1 +do_esp ${ns[4]} dead:3::10 dead:3::1 dead:2::/64 dead:1::/64 $SPI2 $SPI1 -do_dummies4 ns3 -do_dummies6 ns4 +do_dummies4 ${ns[3]} +do_dummies6 ${ns[4]} -do_esp_policy_get_check ns3 10.0.1.0/24 10.0.2.0/24 -do_esp_policy_get_check ns4 10.0.2.0/24 10.0.1.0/24 -do_esp_policy_get_check ns3 dead:1::/64 dead:2::/64 -do_esp_policy_get_check ns4 dead:2::/64 dead:1::/64 +do_esp_policy_get_check ${ns[3]} 10.0.1.0/24 10.0.2.0/24 +do_esp_policy_get_check ${ns[4]} 10.0.2.0/24 10.0.1.0/24 +do_esp_policy_get_check ${ns[3]} dead:1::/64 dead:2::/64 +do_esp_policy_get_check ${ns[4]} dead:2::/64 dead:1::/64 # ping to .254 should use ipsec, exception is not installed. check_xfrm 1 254 @@ -432,11 +432,11 @@ fi # installs exceptions # localip remoteip encryptdst plaindst -do_exception ns3 10.0.3.1 10.0.3.10 10.0.2.253 10.0.2.240/28 -do_exception ns4 10.0.3.10 10.0.3.1 10.0.1.253 10.0.1.240/28 +do_exception ${ns[3]} 10.0.3.1 10.0.3.10 10.0.2.253 10.0.2.240/28 +do_exception ${ns[4]} 10.0.3.10 10.0.3.1 10.0.1.253 10.0.1.240/28 -do_exception ns3 dead:3::1 dead:3::10 dead:2::fd dead:2:f0::/96 -do_exception ns4 dead:3::10 dead:3::1 dead:1::fd dead:1:f0::/96 +do_exception ${ns[3]} dead:3::1 dead:3::10 dead:2::fd dead:2:f0::/96 +do_exception ${ns[4]} dead:3::10 dead:3::1 dead:1::fd dead:1:f0::/96 check_exceptions "exceptions" if [ $? -ne 0 ]; then @@ -444,14 +444,14 @@ if [ $? -ne 0 ]; then fi # insert block policies with adjacent/overlapping netmasks -do_overlap ns3 +do_overlap ${ns[3]} check_exceptions "exceptions and block policies" if [ $? -ne 0 ]; then ret=1 fi -for n in ns3 ns4;do +for n in ${ns[3]} ${ns[4]};do ip -net $n xfrm policy set hthresh4 28 24 hthresh6 126 125 sleep $((RANDOM%5)) done @@ -459,19 +459,19 @@ done check_exceptions "exceptions and block policies after hresh changes" # full flush of policy db, check everything gets freed incl. internal meta data -ip -net ns3 xfrm policy flush +ip -net ${ns[3]} xfrm policy flush -do_esp_policy ns3 10.0.3.1 10.0.3.10 10.0.1.0/24 10.0.2.0/24 -do_exception ns3 10.0.3.1 10.0.3.10 10.0.2.253 10.0.2.240/28 +do_esp_policy ${ns[3]} 10.0.3.1 10.0.3.10 10.0.1.0/24 10.0.2.0/24 +do_exception ${ns[3]} 10.0.3.1 10.0.3.10 10.0.2.253 10.0.2.240/28 # move inexact policies to hash table -ip -net ns3 xfrm policy set hthresh4 16 16 +ip -net ${ns[3]} xfrm policy set hthresh4 16 16 sleep $((RANDOM%5)) check_exceptions "exceptions and block policies after hthresh change in ns3" # restore original hthresh settings -- move policies back to tables -for n in ns3 ns4;do +for n in ${ns[3]} ${ns[4]};do ip -net $n xfrm policy set hthresh4 32 32 hthresh6 128 128 sleep $((RANDOM%5)) done @@ -479,8 +479,8 @@ check_exceptions "exceptions and block policies after htresh change to normal" check_hthresh_repeat "policies with repeated htresh change" -check_random_order ns3 "policies inserted in random order" +check_random_order ${ns[3]} "policies inserted in random order" -for i in 1 2 3 4;do ip netns del ns$i;done +cleanup_ns $ns1 $ns2 $ns3 $ns4 exit $ret diff --git a/tools/testing/selftests/netfilter/.gitignore b/tools/testing/selftests/netfilter/.gitignore index 4b2928e1c19d..c2229b3e40d4 100644 --- a/tools/testing/selftests/netfilter/.gitignore +++ b/tools/testing/selftests/netfilter/.gitignore @@ -2,3 +2,5 @@ nf-queue connect_close audit_logread +conntrack_dump_flush +sctp_collision diff --git a/tools/testing/selftests/netfilter/Makefile b/tools/testing/selftests/netfilter/Makefile index bced422b78f7..db27153eb4a0 100644 --- a/tools/testing/selftests/netfilter/Makefile +++ b/tools/testing/selftests/netfilter/Makefile @@ -14,6 +14,7 @@ HOSTPKG_CONFIG := pkg-config CFLAGS += $(shell $(HOSTPKG_CONFIG) --cflags libmnl 2>/dev/null) LDLIBS += $(shell $(HOSTPKG_CONFIG) --libs libmnl 2>/dev/null || echo -lmnl) -TEST_GEN_FILES = nf-queue connect_close audit_logread sctp_collision +TEST_GEN_FILES = nf-queue connect_close audit_logread sctp_collision \ + conntrack_dump_flush include ../lib.mk diff --git a/tools/testing/selftests/netfilter/conntrack_dump_flush.c b/tools/testing/selftests/netfilter/conntrack_dump_flush.c new file mode 100644 index 000000000000..f18c6db13bbf --- /dev/null +++ b/tools/testing/selftests/netfilter/conntrack_dump_flush.c @@ -0,0 +1,430 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE + +#include <time.h> +#include <libmnl/libmnl.h> +#include <netinet/ip.h> + +#include <linux/netlink.h> +#include <linux/netfilter/nfnetlink.h> +#include <linux/netfilter/nfnetlink_conntrack.h> +#include <linux/netfilter/nf_conntrack_tcp.h> +#include "../kselftest_harness.h" + +#define TEST_ZONE_ID 123 +#define CTA_FILTER_F_CTA_TUPLE_ZONE (1 << 2) + +static int reply_counter; + +static int build_cta_tuple_v4(struct nlmsghdr *nlh, int type, + uint32_t src_ip, uint32_t dst_ip, + uint16_t src_port, uint16_t dst_port) +{ + struct nlattr *nest, *nest_ip, *nest_proto; + + nest = mnl_attr_nest_start(nlh, type); + if (!nest) + return -1; + + nest_ip = mnl_attr_nest_start(nlh, CTA_TUPLE_IP); + if (!nest_ip) + return -1; + mnl_attr_put_u32(nlh, CTA_IP_V4_SRC, src_ip); + mnl_attr_put_u32(nlh, CTA_IP_V4_DST, dst_ip); + mnl_attr_nest_end(nlh, nest_ip); + + nest_proto = mnl_attr_nest_start(nlh, CTA_TUPLE_PROTO); + if (!nest_proto) + return -1; + mnl_attr_put_u8(nlh, CTA_PROTO_NUM, 6); + mnl_attr_put_u16(nlh, CTA_PROTO_SRC_PORT, htons(src_port)); + mnl_attr_put_u16(nlh, CTA_PROTO_DST_PORT, htons(dst_port)); + mnl_attr_nest_end(nlh, nest_proto); + + mnl_attr_nest_end(nlh, nest); +} + +static int build_cta_tuple_v6(struct nlmsghdr *nlh, int type, + struct in6_addr src_ip, struct in6_addr dst_ip, + uint16_t src_port, uint16_t dst_port) +{ + struct nlattr *nest, *nest_ip, *nest_proto; + + nest = mnl_attr_nest_start(nlh, type); + if (!nest) + return -1; + + nest_ip = mnl_attr_nest_start(nlh, CTA_TUPLE_IP); + if (!nest_ip) + return -1; + mnl_attr_put(nlh, CTA_IP_V6_SRC, sizeof(struct in6_addr), &src_ip); + mnl_attr_put(nlh, CTA_IP_V6_DST, sizeof(struct in6_addr), &dst_ip); + mnl_attr_nest_end(nlh, nest_ip); + + nest_proto = mnl_attr_nest_start(nlh, CTA_TUPLE_PROTO); + if (!nest_proto) + return -1; + mnl_attr_put_u8(nlh, CTA_PROTO_NUM, 6); + mnl_attr_put_u16(nlh, CTA_PROTO_SRC_PORT, htons(src_port)); + mnl_attr_put_u16(nlh, CTA_PROTO_DST_PORT, htons(dst_port)); + mnl_attr_nest_end(nlh, nest_proto); + + mnl_attr_nest_end(nlh, nest); +} + +static int build_cta_proto(struct nlmsghdr *nlh) +{ + struct nlattr *nest, *nest_proto; + + nest = mnl_attr_nest_start(nlh, CTA_PROTOINFO); + if (!nest) + return -1; + + nest_proto = mnl_attr_nest_start(nlh, CTA_PROTOINFO_TCP); + if (!nest_proto) + return -1; + mnl_attr_put_u8(nlh, CTA_PROTOINFO_TCP_STATE, TCP_CONNTRACK_ESTABLISHED); + mnl_attr_put_u16(nlh, CTA_PROTOINFO_TCP_FLAGS_ORIGINAL, 0x0a0a); + mnl_attr_put_u16(nlh, CTA_PROTOINFO_TCP_FLAGS_REPLY, 0x0a0a); + mnl_attr_nest_end(nlh, nest_proto); + + mnl_attr_nest_end(nlh, nest); +} + +static int conntrack_data_insert(struct mnl_socket *sock, struct nlmsghdr *nlh, + uint16_t zone) +{ + char buf[MNL_SOCKET_BUFFER_SIZE]; + struct nlmsghdr *rplnlh; + unsigned int portid; + int err, ret; + + portid = mnl_socket_get_portid(sock); + + ret = build_cta_proto(nlh); + if (ret < 0) { + perror("build_cta_proto"); + return -1; + } + mnl_attr_put_u32(nlh, CTA_TIMEOUT, htonl(20000)); + mnl_attr_put_u16(nlh, CTA_ZONE, htons(zone)); + + if (mnl_socket_sendto(sock, nlh, nlh->nlmsg_len) < 0) { + perror("mnl_socket_sendto"); + return -1; + } + + ret = mnl_socket_recvfrom(sock, buf, MNL_SOCKET_BUFFER_SIZE); + if (ret < 0) { + perror("mnl_socket_recvfrom"); + return ret; + } + + ret = mnl_cb_run(buf, ret, nlh->nlmsg_seq, portid, NULL, NULL); + if (ret < 0) { + if (errno == EEXIST) { + /* The entries are probably still there from a previous + * run. So we are good + */ + return 0; + } + perror("mnl_cb_run"); + return ret; + } + + return 0; +} + +static int conntrack_data_generate_v4(struct mnl_socket *sock, uint32_t src_ip, + uint32_t dst_ip, uint16_t zone) +{ + char buf[MNL_SOCKET_BUFFER_SIZE]; + struct nlmsghdr *nlh; + struct nfgenmsg *nfh; + int ret; + + nlh = mnl_nlmsg_put_header(buf); + nlh->nlmsg_type = (NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_NEW; + nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | + NLM_F_ACK | NLM_F_EXCL; + nlh->nlmsg_seq = time(NULL); + + nfh = mnl_nlmsg_put_extra_header(nlh, sizeof(struct nfgenmsg)); + nfh->nfgen_family = AF_INET; + nfh->version = NFNETLINK_V0; + nfh->res_id = 0; + + ret = build_cta_tuple_v4(nlh, CTA_TUPLE_ORIG, src_ip, dst_ip, 12345, 443); + if (ret < 0) { + perror("build_cta_tuple_v4"); + return ret; + } + ret = build_cta_tuple_v4(nlh, CTA_TUPLE_REPLY, dst_ip, src_ip, 443, 12345); + if (ret < 0) { + perror("build_cta_tuple_v4"); + return ret; + } + return conntrack_data_insert(sock, nlh, zone); +} + +static int conntrack_data_generate_v6(struct mnl_socket *sock, + struct in6_addr src_ip, + struct in6_addr dst_ip, + uint16_t zone) +{ + char buf[MNL_SOCKET_BUFFER_SIZE]; + struct nlmsghdr *nlh; + struct nfgenmsg *nfh; + int ret; + + nlh = mnl_nlmsg_put_header(buf); + nlh->nlmsg_type = (NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_NEW; + nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | + NLM_F_ACK | NLM_F_EXCL; + nlh->nlmsg_seq = time(NULL); + + nfh = mnl_nlmsg_put_extra_header(nlh, sizeof(struct nfgenmsg)); + nfh->nfgen_family = AF_INET6; + nfh->version = NFNETLINK_V0; + nfh->res_id = 0; + + ret = build_cta_tuple_v6(nlh, CTA_TUPLE_ORIG, src_ip, dst_ip, + 12345, 443); + if (ret < 0) { + perror("build_cta_tuple_v6"); + return ret; + } + ret = build_cta_tuple_v6(nlh, CTA_TUPLE_REPLY, dst_ip, src_ip, + 12345, 443); + if (ret < 0) { + perror("build_cta_tuple_v6"); + return ret; + } + return conntrack_data_insert(sock, nlh, zone); +} + +static int count_entries(const struct nlmsghdr *nlh, void *data) +{ + reply_counter++; +} + +static int conntracK_count_zone(struct mnl_socket *sock, uint16_t zone) +{ + char buf[MNL_SOCKET_BUFFER_SIZE]; + struct nlmsghdr *nlh, *rplnlh; + struct nfgenmsg *nfh; + struct nlattr *nest; + unsigned int portid; + int err, ret; + + portid = mnl_socket_get_portid(sock); + + nlh = mnl_nlmsg_put_header(buf); + nlh->nlmsg_type = (NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_GET; + nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_DUMP; + nlh->nlmsg_seq = time(NULL); + + nfh = mnl_nlmsg_put_extra_header(nlh, sizeof(struct nfgenmsg)); + nfh->nfgen_family = AF_UNSPEC; + nfh->version = NFNETLINK_V0; + nfh->res_id = 0; + + mnl_attr_put_u16(nlh, CTA_ZONE, htons(zone)); + + ret = mnl_socket_sendto(sock, nlh, nlh->nlmsg_len); + if (ret < 0) { + perror("mnl_socket_sendto"); + return ret; + } + + reply_counter = 0; + ret = mnl_socket_recvfrom(sock, buf, MNL_SOCKET_BUFFER_SIZE); + while (ret > 0) { + ret = mnl_cb_run(buf, ret, nlh->nlmsg_seq, portid, + count_entries, NULL); + if (ret <= MNL_CB_STOP) + break; + + ret = mnl_socket_recvfrom(sock, buf, MNL_SOCKET_BUFFER_SIZE); + } + if (ret < 0) { + perror("mnl_socket_recvfrom"); + return ret; + } + + return reply_counter; +} + +static int conntrack_flush_zone(struct mnl_socket *sock, uint16_t zone) +{ + char buf[MNL_SOCKET_BUFFER_SIZE]; + struct nlmsghdr *nlh, *rplnlh; + struct nfgenmsg *nfh; + struct nlattr *nest; + unsigned int portid; + int err, ret; + + portid = mnl_socket_get_portid(sock); + + nlh = mnl_nlmsg_put_header(buf); + nlh->nlmsg_type = (NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_DELETE; + nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK; + nlh->nlmsg_seq = time(NULL); + + nfh = mnl_nlmsg_put_extra_header(nlh, sizeof(struct nfgenmsg)); + nfh->nfgen_family = AF_UNSPEC; + nfh->version = NFNETLINK_V0; + nfh->res_id = 0; + + mnl_attr_put_u16(nlh, CTA_ZONE, htons(zone)); + + ret = mnl_socket_sendto(sock, nlh, nlh->nlmsg_len); + if (ret < 0) { + perror("mnl_socket_sendto"); + return ret; + } + + ret = mnl_socket_recvfrom(sock, buf, MNL_SOCKET_BUFFER_SIZE); + if (ret < 0) { + perror("mnl_socket_recvfrom"); + return ret; + } + + ret = mnl_cb_run(buf, ret, nlh->nlmsg_seq, portid, NULL, NULL); + if (ret < 0) { + perror("mnl_cb_run"); + return ret; + } + + return 0; +} + +FIXTURE(conntrack_dump_flush) +{ + struct mnl_socket *sock; +}; + +FIXTURE_SETUP(conntrack_dump_flush) +{ + struct in6_addr src, dst; + int ret; + + self->sock = mnl_socket_open(NETLINK_NETFILTER); + if (!self->sock) { + perror("mnl_socket_open"); + exit(EXIT_FAILURE); + } + + if (mnl_socket_bind(self->sock, 0, MNL_SOCKET_AUTOPID) < 0) { + perror("mnl_socket_bind"); + exit(EXIT_FAILURE); + } + + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID); + if (ret < 0 && errno == EPERM) + SKIP(return, "Needs to be run as root"); + else if (ret < 0 && errno == EOPNOTSUPP) + SKIP(return, "Kernel does not seem to support conntrack zones"); + + ret = conntrack_data_generate_v4(self->sock, 0xf0f0f0f0, 0xf1f1f1f1, + TEST_ZONE_ID); + EXPECT_EQ(ret, 0); + ret = conntrack_data_generate_v4(self->sock, 0xf2f2f2f2, 0xf3f3f3f3, + TEST_ZONE_ID + 1); + EXPECT_EQ(ret, 0); + ret = conntrack_data_generate_v4(self->sock, 0xf4f4f4f4, 0xf5f5f5f5, + TEST_ZONE_ID + 2); + EXPECT_EQ(ret, 0); + + src = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x01000000 + } + }}; + dst = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x02000000 + } + }}; + ret = conntrack_data_generate_v6(self->sock, src, dst, + TEST_ZONE_ID); + EXPECT_EQ(ret, 0); + src = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x03000000 + } + }}; + dst = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x04000000 + } + }}; + ret = conntrack_data_generate_v6(self->sock, src, dst, + TEST_ZONE_ID + 1); + EXPECT_EQ(ret, 0); + src = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x05000000 + } + }}; + dst = (struct in6_addr) {{ + .__u6_addr32 = { + 0xb80d0120, + 0x00000000, + 0x00000000, + 0x06000000 + } + }}; + ret = conntrack_data_generate_v6(self->sock, src, dst, + TEST_ZONE_ID + 2); + EXPECT_EQ(ret, 0); + + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID); + EXPECT_GE(ret, 2); + if (ret > 2) + SKIP(return, "kernel does not support filtering by zone"); +} + +FIXTURE_TEARDOWN(conntrack_dump_flush) +{ +} + +TEST_F(conntrack_dump_flush, test_dump_by_zone) +{ + int ret; + + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID); + EXPECT_EQ(ret, 2); +} + +TEST_F(conntrack_dump_flush, test_flush_by_zone) +{ + int ret; + + ret = conntrack_flush_zone(self->sock, TEST_ZONE_ID); + EXPECT_EQ(ret, 0); + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID); + EXPECT_EQ(ret, 0); + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID + 1); + EXPECT_EQ(ret, 2); + ret = conntracK_count_zone(self->sock, TEST_ZONE_ID + 2); + EXPECT_EQ(ret, 2); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh index 92743980e553..a28c1416cb89 100755 --- a/tools/testing/selftests/run_kselftest.sh +++ b/tools/testing/selftests/run_kselftest.sh @@ -20,11 +20,13 @@ usage() { cat <<EOF Usage: $0 [OPTIONS] - -s | --summary Print summary with detailed log in output.log + -s | --summary Print summary with detailed log in output.log (conflict with -p) + -p | --per_test_log Print test log in /tmp with each test name (conflict with -s) -t | --test COLLECTION:TEST Run TEST from COLLECTION -c | --collection COLLECTION Run all tests from COLLECTION -l | --list List the available collection:test entries -d | --dry-run Don't actually run any tests + -n | --netns Run each test in namespace -h | --help Show this usage info -o | --override-timeout Number of seconds after which we timeout EOF @@ -41,6 +43,9 @@ while true; do logfile="$BASE_DIR"/output.log cat /dev/null > $logfile shift ;; + -p | --per-test-log) + per_test_logging=1 + shift ;; -t | --test) TESTS="$TESTS $2" shift 2 ;; @@ -53,6 +58,9 @@ while true; do -d | --dry-run) dryrun="echo" shift ;; + -n | --netns) + RUN_IN_NETNS=1 + shift ;; -o | --override-timeout) kselftest_override_timeout="$2" shift 2 ;; diff --git a/tools/testing/selftests/tc-testing/config b/tools/testing/selftests/tc-testing/config index 012aa33b341b..c60acba951c2 100644 --- a/tools/testing/selftests/tc-testing/config +++ b/tools/testing/selftests/tc-testing/config @@ -82,7 +82,6 @@ CONFIG_NET_ACT_GACT=m CONFIG_GACT_PROB=y CONFIG_NET_ACT_MIRRED=m CONFIG_NET_ACT_SAMPLE=m -CONFIG_NET_ACT_IPT=m CONFIG_NET_ACT_NAT=m CONFIG_NET_ACT_PEDIT=m CONFIG_NET_ACT_SIMP=m diff --git a/tools/testing/selftests/tc-testing/tdc.sh b/tools/testing/selftests/tc-testing/tdc.sh index 407fa53822a0..c53ede8b730d 100755 --- a/tools/testing/selftests/tc-testing/tdc.sh +++ b/tools/testing/selftests/tc-testing/tdc.sh @@ -20,7 +20,6 @@ try_modprobe act_ct try_modprobe act_ctinfo try_modprobe act_gact try_modprobe act_gate -try_modprobe act_ipt try_modprobe act_mirred try_modprobe act_mpls try_modprobe act_nat |