Hi just want to post something to track my PKS upgrade from 1.1.4 to 1.2, so let me give some context here:
PKS on this time was installed in the 1.1.4 version with NSX-T 2.1, nothing weirdo so the deal was to move to 1.2 the PKS, and here is the challenge faced, there is an issue in PKS regarding that version in where some elements are not able to be "deleted", so let me explain more in detail.
PKS is a solution composed of mainly 3 elements: PKS tile, Operations Manager Tile and Bosh Director, extras you have NSX-T and Harbor if registry selection is that, so in short PKS does on-prem a GCP for K8s and consequently for CNA's, so by means of BOSH connects to vSphere and do the magic to create VMs (vSphere based in this case )when you request the creation of PODs at K8s level, so K8`s orchestrates Docker and PKS takes care of K8s and BOSH of the VM´s n a declarative way of construct infrastructure,so long story short when you install everything and integrated with NSX-T PKS creates a superuser in NSX manager to wire all the pods in K8's in automatic was every time you need a new namespace, so if you see any installation include the HOL´s PKS you will see something similar to the pic below.
Fig1. The object created from PKS in NSX-T (note the "key" icon this means that is not possible to modify/deleted by admin )
in this case, everything is fine that is the way PKS connects with NSX-T and tell him you know what? create a Logical Switch and a T0 and wire them and create a Load Balancer and wire it as well and so on so fort you get the idea.
Well in the case you want to upgrade the procedure to follows is documented in pivotal docs (RTM) so first, upgrade NSX-T and everything is OK, but then I have to upgrade a couple of PKS clusters so when you run the upgrade of PKS (at this point everything else you where able / maybe not to upgrade) the cluster send error in bosh cli (https://bosh.io/docs/cli-v2/ ) task will show something like this(my first problem) :
So I saw something like above then I figure out to solve it if anyone has the interest I follow the steps to "touch" the MariaDB for clean my clusters.
Then when I got back to NSX-T there was a lot of things that I don`t need anymore since some clusters where created and have to reference to those objects in PKS (aka PKS create LS in NSX-T) so I went to the UI and try to delete them but them (my second problem) the Edit & Delete option was grayed, so that means that I can do anything with those.
Fig2 Grayed EDIT and DELETE options in NSX-T
just trying to get the most easy way to solve this 'I asked' since everyone talks about the problem but just after a while somebody told me you have to follow the yellow brick road, so at the end I was pointed to became superuser (PKS superuser) and then delete the objects, so I find the instructions but there where like many steps from the perspective of doing many steps and with high risk if you miss one letter or copy paste certificate for the user etc, a nightmare inside a nightmare so please don't go there.
So I find this link and after testing the procedure and easy API call (Deletetion in this case) http://keithlee.ie/2018/12/27/how-to-delete-nsx-t-protected-objects/ (Kudos to this guy) or set of API calls wich seem more simple, in additon or contribution to that I modeify the API calls usign like this:
*****************************************************************************************************************************
curl -k -X GET "https://${NSX_MANAGER_IP}/api/v1/logical-switches" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD"
***************************************************************************************************************************************
Delete Logical Switch
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/logical-switches/3f3d655b-32a5-493e-8f4c-1355df133a93?detach=true&cascade=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
**************************************************************************************************************************
Delete Logical Router
curl -k -u admin:Passw0rd.2018 -X DELETE ‘https://10.235.160.75/api/v1/logical-routers/
38218e39-a10f-4e7c-a7f8-46610ceb4798?force=true’ -H “X-Allow-Overwrite: true”
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/logical-routers/29b091f3-7c93-49d0-960f-430e0913f488?force=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
************************************************************
Delete Logical Router Router Ports
curl -k -u admin:Passw0rd.2018 -X DELETE ‘https://10.235.160.75/api/v1/logical-router-ports/0806b919-201a-467b-8ccc-820a02b4ca31?force=true’ -H “X-Allow-Overwrite: true”
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/logical-router-ports/0806b919-201a-467b-8ccc-820a02b4ca31?force=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
******************************************************************************************
Delete NAT Rules
curl -k -u admin
:Passw0rd
.2018 -X DELETE ‘https://10.235.160.75/api/v1/logical-routers/6585c274-b395-4189-a7b6-c30dc6c38d4d/nat/rules/2064’ -H “X-Allow-Overwrite: true”
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/logical-routers/6585c274-b395-4189-a7b6-c30dc6c38d4d/nat/rules/2064" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
*************************
Delete IPAM Subnet
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/pools/ip-subnets/422bd4bd-3b21-423f-a7eb-818b15c3ada9" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
****************
Delete IP Pools
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.kie/api/v1/pools/ip-pools/4e6bfe2a-a4f9-4701-9756-79608ad10d61?force=true’ -H “X-Allow-Overwrite: true”
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/pools/ip-pools/05586755-0bff-4cb4-afcd-6ba03d8bd87a?force=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
*************************
Release IP from IP Pool
curl -k -u admin:Passw0rd.2018 -X POST ‘https://10.235.160.75/api/v1/pools/ip-pools/d11c1c51-ff8f-45cd-b4ab-bed93cf8a02d?action=RELEASE’ -H “X-Allow-Overwrite: true” -d ‘{“allocation_id”:”10.0.80.11″}’ -H “Content-Type: application/json”
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/pools/ip-pools/XX?action=RELEASE" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true" -d '{"allocation_id":"10.0.80.11"}' -H "Content-Type: application/json"
read them
curl -k -X GET "https://${NSX_MANAGER_IP}/api/v1/pools/ip-pools/" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD"
******************************
Delete Load Balance"
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/loadbalancer/services/4c90ca51-5249-48cf-aaf9-d431833c0039" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
********************
Delete Virtual Servers
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/loadbalancer/virtual-servers/af6da6a1-db09-4100-abd9-1276c6a48b3a?delete_associated_rules=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
******************************
Delete Load Balancer Pools
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/loadbalancer/pools/c69b66be-fb4c-4164-85bd-72441a525c0b" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
************************************************************
Delete Load Balancer Monitor
curl -k -X DELETE "https://${NSX_MANAGER_IP}/api/v1/loadbalancer/monitors/8907ef61-8d8d-5dbd-96af-45b8dafa6627" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
******************************************************************************************
Delete Group
curl -k -X DELETE ‘https://${NSX_MANAGER_IP}/api/v1/ns-groups/e80abffa-26f6-4de9-a419-0fb0505ef316?force=true" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" -H "X-Allow-Overwrite: true"
*****************************************************************************************************************************
I did an export for the $NSX_MANAGER_USERNAME and $NSX_MANAGER_PASSWORD as well for the ${NSX_MANAGER_IP} to make them easy to read on my terminal
Fig4 CLI API Call to DELETE in this case the LS
Fig5 After execution of API call the LS is deleted
Finally, I could continue to my upgrade after feeling the pain I just want to post this for somebody to find it useful, I spend like 8 hours on this so maybe this can save you time and headaches.
Note: You may have something similar but at the end, the problem will arise and you will have to spend a long time checking about it, so the symptom at this point is a) you need to upgrade PKS and b) you can´t "clean" objects from previous installation, and here you will see if you are finishing the deployment and then you want to leave as it was installed by the first time or like in my case need to upgrade and in order to upgrade I was unable to have a clean update since objects were not possible to remove from NSX-T.
Thanks for reading
+vRay