• About Dangerous DBA
  • Table of Contents
Dangerous DBA A blog for those DBA's who live on the edge

Category Archives: Cloud

Google Cloud Management – My Idea – My White Whale?

May 20, 2020 1:00 pm / Leave a Comment / dangerousDBA

Synopsis:

Google Cloud Platform has awesome built in functionality for keeping your cloud projects secure. Microsoft Active directory and LDAP like services are used by enterprises to manage the users accessing the (IT) resources of the business. Google Cloud Platform requires a user to have a Cloud Identity to be able to access the resources of a project, be that GSuite or from another source such as a federated Active Directory

Problem:

From what I have experienced, seen and would love to know how other people deal with this is that because the resources of a project are managed by a different part of the organisation than the Active Directory, and the resources might be managed at a different “lower” level again e.g.

  • “IT” manage the creation of a GSuite account (if you have it) and they also create separately an Active Directory (or equiv) to allow the machine (and user) access to the network, printers, drives etc. The user is also more than likely put into one or more “groups”
  • Some form of project admin for your Google Cloud Projects has to grant the user (via the user name or the group they belong too) access to the project and potentially resources in the project.
  • Over time, and due to legal or contractual requirements users may have their rights to use resources changed and this needs to be easily managed and not be a conglomeration of two teams time to work out who job it is to revoke the rights.

Added to this to manage these requests you may use any number of systems to grant guest access or other rights to resources.

Solution:

I am proposing to (at a super high level) build a system that will:

  1. Standardise the naming conventions of the groups an organisation could have
  2. Have a Meta datastore that holds the organisational structure of groups, users (physical and IAM) and projects permissions
  3. A process that runs in a project and queries the Meta datastore to get the permissions that should be in place in the requesting project and set them
  4. A store of the history of the changes that have been made to the permissions.

Where to start:

I am first of all going to look at how Google cloud projects are set up and what the hierarchy of resource, are the permissions uniform (analogous) across all types of resource is and progress from there.

FULL DISCLOSURE: I have already made a start here, but knowing what I do now then I am going to start again as I think there are improvements to be made. Also Google are always updating what they do and therefore things have changed!

Posted in: 2020, AWS, Cloud, GCP, Google

Position Tracker – In the beginning

May 13, 2020 9:00 am / Leave a Comment / dangerousDBA

As mentioned in the previous post starting this all up again then I have been looking at expanding and improving my skills around python and Google Cloud.

What did I do:

To this end I took up Googles offer and signed up for the “Free courses during COVID” offer and followed through with the QwickLabs modules. One of the quests contained a IOT device simulator, this was a fairly simple amount of data that was passed through all the way to Google Big Query. I am going to take this and completely plagiarise it as a learning example to build upon as I think it is a good basis for a lot of things:

  1. Improving my python – The first iteration I will publish I plan to replace the data files the code ingests with something that generates “random” data. There is a lot of scope to use different methods to improve this
  2. Extending the pipeline – This pipeline can go all the way through to visualisation in Google Data Studio
  3. Looking at Google Big Query – This is a very interesting area, we can look at functions, GIS and all things GBQ
  4. Other Google Services – There are many services used in this example and I feel that we can add more as we need such as Google Cloud Composer,

Where can this wonder code be found:

I have a Git Hub account where I have various white whales that I have started, and this particular one can be found: here

How is this going to work:

I am going to start by creating a pipeline that is not much of a departure from what is offered by the current Quest. I will then iterate on that to produce proof of concepts and give appraisals of what I have done, try and critique myself. You will be able to find the work in my GitHub repo and we can see where we go from here, depending on mainly when I get time to do these things!

Who can help:

You all can if you think that there is a better way to do literally everything I would love to know and investigate. I am pretty certain that there is for my Python location data generating stub after the first rushed iteration.

I look forward to hearing from you all!

Posted in: 2020, Big Data, BigQuery, BigQuery, Cloud, Dataflow, GCP, Google, IOT, Position Tracker, PubSub / Tagged: BigQuery, Dataflow, Devices, GCP, Google, Google Cloud, IOT, Position Tracking, PubSub

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 757 other subscribers

Recent Posts

  • Self generating Simple SQL procedures – MySQL
  • Google Cloud Management – My Idea – My White Whale?
  • Position Tracker – The Stub – Pandas:
  • Position Tracker – The Stub
  • Position Tracker – In the beginning
  • Whats been going on in the world of the Dangerous DBA:
  • QCon London Day 1
  • Testing Amazon Redshift: Distribution keys and styles
  • Back to dangerous blogging
  • DB2 10.1 LUW Certification 611 notes 1 : Physical Design

Dangerous Topics

added functionality ADMIN_EST_INLINE_LENGTH Bootcamp colum convert data types DB2 db2 DB2 Administration DB2 Development db2advis db2licm Decompose XML EXPORT GCP Google IBM IBM DB2 LUW idug information centre infosphere IOT LOAD merry christmas and a happy new year Position Tracking python Recursive Query Recursive SQL Reorganisation Reorganise Reorganise Indexes Reorganise Tables Runstats sql statement Stored Procedures SYSPROC.ADMIN_CMD Time UDF User Defined Functions V9.7 V10.1 Varchar XML XML PATH XMLTABLE

DangerousDBA Links

  • DB2 for WebSphere Commerce
  • My Personal Blog

Disclaimer:

The posts here represent my personal views and not those of my employer. Any technical advice or instructions are based on my own personal knowledge and experience, and should only be followed by an expert after a careful analysis. Please test any actions before performing them in a critical or nonrecoverable environment. Any actions taken based on my experiences should be done with extreme caution. I am not responsible for any adverse results. DB2 is a trademark of IBM. I am not an employee or representative of IBM.

Advertising

© Copyright 2022 - Dangerous DBA
Infinity Theme by DesignCoral / WordPress